XM for Strategy & Research Reviews

“More flexible than competitors”

Pros

Compared with the other survey tools I've used, like SurveyMonkey, Wufoo and Google Forms, this one gives me the most flexibility and functionality while still keeping the interface user-friendly. Creating surveys is pretty painless and whenever I'm building something more complex, I never think twice about contacting support because the wait times are short and the help is genuinely detailed and useful. It lets our lab build, share and score surveys efficiently. The Library feature is especially valuable since we can save surveys, blocks and questions there, then copy, edit or reuse them in later projects. We use both personal libraries and a shared lab library. The scoring tools alone save us days of data cleaning. We're able to score the same survey multiple times for different categories we set up, like Depression Score or a Hopelessness Subscale Score. It also allows us to customize point values for question items which matters a lot when a Likert scale doesn't run 1-5 or when an item needs reverse coding. For more advanced or customizable calculations, we rely on embedded data such as converting total scores into word labels based on cutoffs. Because of these scoring options, exported data is usually ready for analysis with very little cleanup needed. And before getting into the downsides, I should say the support team is consistently open to feedback and has even contacted me directly to ask what features I'd want added.

Cons

That said the scoring feature, while helpful, is also complex and confusing. It doesn't automatically assign point values based on how an item is recoded so questions have to be reverse coded both during setup and again within scoring. Another frustration is that scoring settings don't carry over with a survey when it's added to the library. I've also run into repeated glitches where scoring wouldn't let me create new categories and even after more than eight months, support still hasn't come back with a solution. Embedded data creates its own set of problems too. Since it's case sensitive, you have to be extremely careful when entering it into logic or piped text. I've had logic fail or data come out wrong over something as small as an extra space at the end of a field. It would help a lot if Qualtrics let users select embedded data fields from a dropdown instead of typing them manually. I've also had bugs where certain dropdown options in Survey Flow wouldn't appear and support hasn't been able to explain that either. Survey Flow itself feels complex, especially in larger surveys with a lot of branching, logic and embedded data. It can be difficult to navigate and it really needs a sidebar showing levels to make management easier. A built-in checker for incorrect logic or embedded data would also save a lot of trial and error. Other improvements I'd still like to see are conditional formatting for email triggers, the ability to break down survey reports by individual respondent and a way to connect survey reports so data can be tracked over time, like weekly anxiety symptom reports.