From: Felipe Contreras < felipe contreras gmail com> To: desktop-devel-list < desktop-devel-list gnome org> Subject: GNOME user survey 2011 (v5) Date: Tue, 6 Sep 2011 23:38:09 +0300 Hi, Here's the next version of the survey. Unfortunately, it seems this is not going to be blessed by GNOME, and questionpro.com only allows 10 questions in the free version. I haven't found a better free online survey, and unless somebody offers hosting for this survey, it would have to be limited. Now, there has been a misconception that I am not listening to some feedback, which is not true. From my point of view there have been two kinds of feedback: a) Criticism to improve the survey All these comments have been addressed in one form or another. Unfortunately, I am forced to drop some of the suggested questions to fit the 10 question limit. Further criticism is still welcome. b) Criticism without possibility for improvements Many people in this category have already accepted that they don't think an online user survey is actually possible, so there's really no way to address these. One problem raised was the issue of self-selection bias, of course, without any suggestions to get rid of it. All online surveys have this issue, however, my proposal was actively suggest people to push others to take the survey (those people would not be self-selected), and have a question to identify them. Then we can ignore the self-selected people, or have two analyses. Of course, I didn't get any response. In a private discussion, Matthew Garret pointed out that there's still the issue that some people might not have friends, and thus there wouldn't be people to push them to take the survey. While this is true, this is something not even professionals can overcome, the only thing possible is to make a guess about the amount of people in this category, and thus the amount of bias. What about other kinds of bias? Would the survey be invalidated if we missed some group of people? No. There's a process called poststratification that would allow us to make valid guesses. Yes, it might turn out that in the end we don't have enough statistical power to make any conclusions, but we won't be able to say that until we get the results. Another misconception is that this survey might cause some damage, by drawing the wrong conclusions. As I already explained before, raw data is raw data, and it can't cause damage. What can cause damage is to make decisions based on wrong conclusions, which is why GNOME developers might want to ignore the conclusions. Or different people might try to do different post-hoc power analysis to try to get some consensus on what conclusions are actually valid. I will continue to listen for suggestions for improvement, but I fear GNOME developers don't want to continue improving this survey in this mailing list, so feel free to contact me personally if you are interested in v6 (if needed), or follow this: https://gist.github.com/1128166 Cheers. === 01. Which of the following best resemble your desktop? === (click to see the image) (in case you are not sure what GNOME is) [single choice] - Windows http://origin.arstechnica.com/images/windows7/Peek%20-%20Before.png - Mac OS X http://www.guidebookgallery.org/pics/gui/desktop/firstrun/macosx103.png - GNOME 2 http://library.gnome.org/misc/release-notes/2.28/figures/gnome-2.28.png.en - GNOME 3 http://gnome3.org/img/overview-big.png - Unity http://static.arstechnica.com/shell-windows.png - KDE http://www.linuxnov.com/wp-content/uploads/2011/07/KDE-4-7-desktop.jpg - I can't tell === 02. Overall, how satisfied are you with GNOME? === [single choice] * not at all * barely * halfway * mostly * completely === 03. Does GNOME do what you want? === [single choice] * not at all * barely * halfway * mostly * completely === 04. What is the main reason you are taking this survey? === [single choice, with other] * Somebody is pushing me * I want to provide feedback to the project * I feel I have to * Other === 05. How many years of experience do you have using computers? === [numeric] === 06. How often do you use a terminal/console? == [single choice] * What is that? * When I have no other option * I can't live without them * Is there anything else? === 07. Have you contributed to the GNOME project? === [single choice] * Yes * No === 08. Have you contacted the GNOME team? === [single choice] * Yes, successfully * Yes, unsuccessfully * No, I don't know how * No, never had the need === 09. If you could change three things in GNOME, what would they be? === [free form] === 10. Do you have any comments or suggestions for the GNOME team? === [free form] -- Felipe Contreras