Annual OSD survey to better understand and serve our stakeholders #77
Comments
|
Technology
I heard that Lime Survey needs relatively attentive administration, since there are relatively frequent security problems (hard to assess for me, but the answer came up already several times) Libreoffice/@HeikoTietze have a Lime Survey installation, they may know more or be even able to "donate" the infrastructure (?) There is also a by-us tool by @simonv3 (?) for surveys. Method I strongly suggest we split the survey according to the groups we want to ask:
We also should prioritize these to know which group to tackle first. I personally think that OSD would be the easiest and we should do it first. I hope for the most interesting outcome with asking Devs but that would be 2nd since it is more complicated, method wise. |
|
Run my first survey with Lime recently. Great flexibility seems to always come with bad usability, and the setup of a test requires to get used to (mostly html/javascript). I don't know how much effort the administration makes, we have an individual for our infrastructure. |
|
@jdittrich, the problem with splitting the survey into stakeholders is that many people are users, developers and designers at the same time. This is why I prefer a more powerful tool that supports what LimeSurvey calls skip logic or branching. Some other survey tools are described here. |
|
@studiospring We can, at the end, combine these again and branch etc. I suggest planning separately: The questions we want to ask and how we ask them might be very different: For developers, I would ask broader, free form questions, more in a qualitative direction; for OSD members we can assume a bit more existing knowledge, so there, more quantitative questions will be asked etc. Like in design (well, it is research design) we should collect our questions and assumptions first. |
|
FWIW - if then statements are not off the books for Quick Survey, I just don't have time to add them. If you know of a Meteor developer who wants to take a stab at it, then they should feel more than welcome. |
|
@jdittrich yes, definitely group related questions together, but ask them in the one survey. @HeikoTietze, when you say LimeSurvey has bad usability, is that for respondents or survey builder? If the survey builder suffers bad usability, do you think the extra features justify the added time and frustration? |
|
Definitely for the survey builder. Guess the system is hard to top for respondents when everything is setup nicely. Hard to say if extra costs are justified in the end. Guess you may become familiar with the system, some people like to fiddle with all the options (including me), and when there is no other choice the answer is clear. |
|
@HeikoTietze I would prefer to make it easier for the respondent by hiding unnecessary questions rather than asking them to check whether it is relevant for them. Hopefully, there will not be too many branches. This will help in improving completion rates. Further, because (I assume) OSD intends to be the go-to authority in open source design, I think we are obliged to set the standard. It also improves our credibility as an organisation. You raise some great points! All else being equal, the richer and more qualitative the data the better. However, we do have to work within the constraints of our research tools and resources. I have not given much thought to the number crunching part of this research. Perhaps this is another reason to do a pilot survey.
I guess if I summarised the survey in to (almost) one question for each stakeholder/role, it would be:
These questions all tie back in to improving the UX of OSS, which is the main goal of OSD. I'm sure all these things will be refined and polished as we go along. This is just a starting point! Perhaps personas are not the best way, but I do want to get inside our stakeholders' heads. Remember personas are more a psychographic profile, less demographic. Maybe I could just put these questions in the survey (more tactfully!), but I think we can tease out a more accurate picture with more subtle questions. I'm happy to use whatever is the best tool, persona or otherwise, to answer these questions. |
I drafted a new issue – make the primary goal of Open Source Design more immediately visible at the home page – but refrained because the existing primary line is subtly different:
– with things such as user experience and interface design further down the page. Re: opensourcedesign/opensourcedesign.github.io#17 (closed) and opensourcedesign/opensourcedesign.github.io#43 (merged) a review of the home page may be timely; should it be made an issue now (in readiness for outcomes of the survey), or later? |
|
@grahamperrin, good point. Consistency is a good thing. I think "great design" and "UX" are largely compatible with each other, so it seems to me to be (merely) semantics. I prefer "UX" because it is clearer and more professional. That said, I don't mind using "great design" from now on in the survey. Is it worth creating an issue to change the home page? Thoughts? |
|
One suggestion for (a) survey(s). I don't have a section for designers, since I was unsure of what we want to know about them (and what of that could be answered well in a survey instead of e.g. talking/focus group/whateverothermethod) Survey part for Developers
part for OSD Community members
|
|
Some first thoughts: Not sure if that's relevant for only me but design is to split into visual design (aethetical presentation mainly) and information architecture (people discuss this term and whether or not it has to get separated further; but in contrast to the visual aspect I see here the general workflow, where functions are placed, how controls are utilized etc.). Asking for design puts me in a difficult situation hence. "In my work I like to" could be "I use to do the following tasks" (Likert from never to always): Documentation, Help (I would split docu and help as the latter is to me what goes into a program; would be interesting if the results show this), Localization, Visual Design (CSS/Styles/Animation), Usability (HIG, Persona etc.), Information architecture (Mockups/Wireframe), Prototyping, Developing, User only (Level could be interesting), and maybe more. UX infrastructure: l10n (Transifex, Pootle etc), VoIP (SIP, Jitsi, Mumble), Email, Forums, Code hosting (Github, Sourceforge...) 1st level support, Regular meetings (weekly, monthly etc.), IRC/Telegram (or any other chat), Donation (Gittip, Paypal etc), official/individual blogs (Planet aggregation). |
Are you referring to the items in the "skillset" part?
It depends on what we want to know – reported emotional stance or reported frequency. I went for the emotions, based on what I read about community motivations (e.g. adding functions is more fun than refactoring)
Do you suggest asking for the infrastructure and if yes, both Devs and OSD members, or just one of them? (I like the idea) |
It starts with "good/bad experiences with design in the past". And my answer would be: depends.
That was your idea :-). I just added some more aspects. |
|
Just letting everyone know that I am aiming to create a survey repo containing an initial outline of the project some time next week, if not earlier. Keeping in mind that the intention is to make this an annual (?) survey, what is an appropriate name for the repo?
Can the name be more specific, in case other surveys are created in future? |
|
I am fine with having a repo (Annual-Stakeholder-Survey?), I assume, though, that the planning of the survey will not happen via pull requests&co; however it might a a good container for the research related discussions (on the other hand, we then have a additional repo which is hard to follow for non-core contribs) |
|
@jdittrich I was thinking of using pull requests and Issues, simply because there were no alternatives. To make it easier, I intend to create a "rough draft" of questions, etc in a markdown document, which will later be coded up. However, since Discourse looks like it is coming online very soon #79 that may be a better platform (hmmm, I'm sure I've said that somewhere before!). I'm obviously open to suggestions. |
Whilst I love clarity and consistency, I don't require things to be defined. I value professionalism, in a be excellent to each other way. I should describe myself as:
… results of a survey should help to untangle contradictions such as those. Towards an understanding of stakeholders; of members of this extraordinarily open GitHub organisation. Sorry. My intention to provide a meaningful answer to the question was overtaken by a sensation of walking through rooms full of these –
– except they're knotted, and some of them have torn off but still, the remnants are over my shoulders and in my face and now I need to sit here –
– and complete a survey 😄 |
As mentioned in issue #73, I would like to start an online survey to learn more about the various stakeholders in OSS so that it easier for OSD to achieve its goal of improving the UX of OSS. Design is not, and has difficulty being, a regular part of OSS development. I approach this problem from a design perspective and the first thing to do as a designer, is understand the problem better. Therefore, with the above goal of OSD in mind, I would like to learn more about the main stakeholders involved. These include:
I would like to gather enough information to be able to:
There are other bits and pieces (nice to haves), but the three points above are probably the meat of the survey. I also mentioned in issue #73 an idea for a usability index. Given the difficulty and the fact that the focus of that research is projects, not people, I do not want to include it as part of this survey (but please think about it!).
Ultimately I want this survey to show us how OSD can serve its stakeholders (not just members!) better.
Some other points:
I have looked at technical details and my first thought is that a self hosted LimeSurvey site (GitHub repo here) is the best tool for the job. Other suggestions, experiences, etc are most welcome.
The other thing we will need is (free) hosting. Suggestions welcome.
The next thing to do, after the high level decisions have been made, is to set up a dedicated repo. Given that a new repo will be set up, please keep this discussion focussed on high level issues and keep details (like survey questions) for the new repo.
Finally, I do not want to do this all myself (especially since PHP and MySQL are not my strengths), so please leave a "thumbs up", so I can gauge contributor interest.
The text was updated successfully, but these errors were encountered: