From: Felipe Contreras < felipe contreras gmail com>
To: desktop-devel-list gnome org
Subject: GNOME user survey 2011
Date: Sun, 31 Jul 2011 19:11:34 +0300
Hi,
Last time I suggested something like this the response was not so
great, but lately I've feeling that there's a lot of dissatisfaction
with GNOME 3. Why not find for good what people are thinking with an
user-survey?
I looked around for different online survey sites and unfortunately
many have many limitations, but questionpro.com seems to have support
for free surveys with unlimited amount of responses, which is
presumably what we would want to get the maximum amount of feedback.
Many of these are borrowed from the Git user survey. The results as
you can see, can be quite interesting:
https://git.wiki.kernel.org/index.php/GitSurvey2010
It would be great if some sort of notification would popup directly on
user's desktops, this way it can ensured that the maximum amount of
people are notified. Otherwise, I think planet GNOME, reddit, twitter,
Google+ and so on should give plenty of feedback. Maybe also contact
Ars Technica, LWN, Phornix, and so on would help.
Here's my current proposal, I tried to make it as small as possible
while keeping important information, it should take a bit more than a
minute to complete. What do you think?
=== 01. Overall, how happy are you with GNOME? ===
(single choice)
* unhappy
* not so happy
* happy
* very happy
* completely ecstatic
=== 02. How long have you been using GNOME? ===
(numeric)
=== 03. How do you describe the amount of configurations available? ===
(single choice)
* More than I need
* Just enough
* Too few
=== 04. Which GNOME version(s) are you using? ===
(multiple choice, with other)
+ 3.0
+ 2.20 - 3.0
+ 2.10 - 2.20
+ 2.0 - 2.10
+ pre 2.0
+ other, please specify
=== 05. How do you compare the current GNOME version with the version
from one year ago? ===
(single choice)
* better
* no changes
* worse
* cannot say
=== 06. What channel(s) do you use to request help about GNOME (if any)? ===
(multiple choice, with other)
+ GNOME bugzilla
+ GNOME mailing list
+ IRC (#gnome)
+ asking colleague/friend
+ instant messaging (IM) like XMPP/Jabber
+ StackOverflow or other StackExchange site
+ N/A (haven't requested help)
+ other (please specify)
=== 07. Does GNOME include code or documentation by you? ===
(single choice)
* Yes
* No
=== 08. Do you have any comments or suggestions for the GNOME team? ===
(free form)
--
Felipe Contreras
From: Olav Vitters < olav vitters nl>
To: Felipe Contreras < felipe contreras gmail com>
Cc: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Sun, 31 Jul 2011 22:14:57 +0200
On Sun, Jul 31, 2011 at 07:11:34PM +0300, Felipe Contreras wrote:
> Many of these are borrowed from the Git user survey. The results as
> you can see, can be quite interesting:
> https://git.wiki.kernel.org/index.php/GitSurvey2010
>
> It would be great if some sort of notification would popup directly on
> user's desktops, this way it can ensured that the maximum amount of
> people are notified. Otherwise, I think planet GNOME, reddit, twitter,
That's impossible.
> Google+ and so on should give plenty of feedback. Maybe also contact
> Ars Technica, LWN, Phornix, and so on would help.
Those are only technical sites. I think the results are going to be
biased whatever you do.
> === 01. Overall, how happy are you with GNOME? ===
I like this one.
> === 02. How long have you been using GNOME? ===
This one too.
> === 03. How do you describe the amount of configurations available? ===
I don't see the relevance of asking this. Furthermore the question is
suggestive. Seems more to prove a point than anything else.
> === 04. Which GNOME version(s) are you using? ===
This should be asked earlier.
> === 05. How do you compare the current GNOME version with the version
> from one year ago? ===
This is vague. Current as in GNOME 3.2 / 3.0, or current as in the
version that they are using. Furthermore, you don't know if they upgrade
each year.
> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
With what purpose is this asked? No support is given on GNOME Bugzilla.
Only minimal on gnome-list.
> === 07. Does GNOME include code or documentation by you? ===
Suggest adding translations as well.
> === 08. Do you have any comments or suggestions for the GNOME team? ===
How will you handle all the various comments you'll get? Git received
9000 answers.. who is going to read them all and summarize?
Things I'd like to know:
- Multiple choice options to understand the responders technical
skills. So if they've submitted a patch, contributed code somewhere,
paid to write software, maintain some open source code, etc.
That is IMO a nicer way to understand their skills can someone to
rate themselves.
- If they use KDE/XFCE/something
- other things to later on be able to determine if the survey is biased
in some way. Don't have concrete ideas atm.
--
Regards,
Olav
From: Felipe Contreras < felipe contreras gmail com>
To: Felipe Contreras < felipe contreras gmail com>, desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 00:16:54 +0300
On Sun, Jul 31, 2011 at 11:14 PM, Olav Vitters < olav vitters nl> wrote:
> On Sun, Jul 31, 2011 at 07:11:34PM +0300, Felipe Contreras wrote:
>> Many of these are borrowed from the Git user survey. The results as
>> you can see, can be quite interesting:
>> https://git.wiki.kernel.org/index.php/GitSurvey2010
>>
>> It would be great if some sort of notification would popup directly on
>> user's desktops, this way it can ensured that the maximum amount of
>> people are notified. Otherwise, I think planet GNOME, reddit, twitter,
>
> That's impossible.
Everything is possible.
>> Google+ and so on should give plenty of feedback. Maybe also contact
>> Ars Technica, LWN, Phornix, and so on would help.
>
> Those are only technical sites. I think the results are going to be
> biased whatever you do.
How do you suggest to reach the end users if you already dismissed a
pop up directly from the desktop? If there isn't any way, at least
it's better than nothing.
>> === 03. How do you describe the amount of configurations available? ===
>
> I don't see the relevance of asking this. Furthermore the question is
> suggestive. Seems more to prove a point than anything else.
I do see the relevance, as I think it has been a big point of
contention raised by many users.
If you don't think it's very important, it could go to the end.
Besides, if the results are overwhelmingly pointing to "Too few"?
Don't you think it would be worth investigating? If the results end up
being "Just enough", then no harm done.
>> === 04. Which GNOME version(s) are you using? ===
>
> This should be asked earlier.
Where?
>> === 05. How do you compare the current GNOME version with the version
>> from one year ago? ===
>
> This is vague. Current as in GNOME 3.2 / 3.0, or current as in the
> version that they are using. Furthermore, you don't know if they upgrade
> each year.
Obviously it's the one they are using, otherwise how would they compare?
Would this help?
How do you compare your current GNOME version with the version from
one year ago?
>> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
>
> With what purpose is this asked? No support is given on GNOME Bugzilla.
> Only minimal on gnome-list.
It's important to have a two-way communication with the users, don't
you think? So it's important to figure out what channels they actually
use. In fact, I was thinking to add another question asking if they
feel there's lack of communication with the team.
Resolving bugs is also support. But feel free to rephrase the question
to find out how users provide any kind feedback; questions, bug
reports, issues, etc.
>> === 08. Do you have any comments or suggestions for the GNOME team? ===
>
> How will you handle all the various comments you'll get? Git received
> 9000 answers.. who is going to read them all and summarize?
I would, if nobody takes the task. Most likely I will miss a lot of
important stuff if I'm the only one. But it is important to ask this,
as the survey is bound to be imperfect, and this is the only place
where people can actually suggest what to add to the next survey
(indirectly).
> Things I'd like to know:
> - Multiple choice options to understand the responders technical
> skills. So if they've submitted a patch, contributed code somewhere,
> paid to write software, maintain some open source code, etc.
> That is IMO a nicer way to understand their skills can someone to
> rate themselves.
How about Zeeshan's suggestion of asking how often they use the terminal?
> - If they use KDE/XFCE/something
That's confusing. You mean if they switch between GNOME and those? Or
that they used to use GNOME, and now they use that one?
How about:
Which other desktop environments have you used?
> - other things to later on be able to determine if the survey is biased
> in some way. Don't have concrete ideas atm.
Well, how are you going to determine if the survey is biased in some
way, or it's the actual GNOME users that are biased in some way?
Either way, I think the only important bias is geekness.
--
Felipe Contreras
From: Olav Vitters < olav vitters nl>
To: Felipe Contreras < felipe contreras gmail com>
Cc: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Sun, 31 Jul 2011 23:49:18 +0200
On Mon, Aug 01, 2011 at 12:16:54AM +0300, Felipe Contreras wrote:
> On Sun, Jul 31, 2011 at 11:14 PM, Olav Vitters < olav vitters nl> wrote:
> > On Sun, Jul 31, 2011 at 07:11:34PM +0300, Felipe Contreras wrote:
> >> Many of these are borrowed from the Git user survey. The results as
> >> you can see, can be quite interesting:
> >> https://git.wiki.kernel.org/index.php/GitSurvey2010
> >>
> >> It would be great if some sort of notification would popup directly on
> >> user's desktops, this way it can ensured that the maximum amount of
> >> people are notified. Otherwise, I think planet GNOME, reddit, twitter,
> >
> > That's impossible.
>
> Everything is possible.
>
> >> Google+ and so on should give plenty of feedback. Maybe also contact
> >> Ars Technica, LWN, Phornix, and so on would help.
> >
> > Those are only technical sites. I think the results are going to be
> > biased whatever you do.
>
> How do you suggest to reach the end users if you already dismissed a
> pop up directly from the desktop? If there isn't any way, at least
> it's better than nothing.
I didn't dismiss. I said it was impossible to popup a notification.
Survey is only good if either the bias can be determined, or if it is
unbiased. As mentioned, I think it will be biased, so the survey should
have some questions to determine the bias.
> >> === 03. How do you describe the amount of configurations available? ===
> >
> > I don't see the relevance of asking this. Furthermore the question is
> > suggestive. Seems more to prove a point than anything else.
>
> I do see the relevance, as I think it has been a big point of
> contention raised by many users.
Something should be done with a survey. No matter the outcome of this
question, you won't be able to take these results and change things.
Asking if people want more configuration options goes against why
options are removed. Ideally everything should happen automatically.
I'm only interested in the cases where it doesn't work.
> If you don't think it's very important, it could go to the end.
> Besides, if the results are overwhelmingly pointing to "Too few"?
> Don't you think it would be worth investigating? If the results end up
> being "Just enough", then no harm done.
The question is suggestive so the results will be biased. It is better
to ask something like "does GNOME do what you want?" and then have a
text field where they can specify what GNOME lacks.
> >> === 04. Which GNOME version(s) are you using? ===
> >
> > This should be asked earlier.
>
> Where?
before #3.
IMO the questions should be ordered so that the basic/factual stuff is
first, possibly ask what they like, then questions where you just say
what you feel, and every improvement suggestion should come last.
> >> === 05. How do you compare the current GNOME version with the version
> >> from one year ago? ===
> >
> > This is vague. Current as in GNOME 3.2 / 3.0, or current as in the
> > version that they are using. Furthermore, you don't know if they upgrade
> > each year.
>
> Obviously it's the one they are using, otherwise how would they compare?
That should be made clear in the question, otherwise it will be
interpreted differently and the results are biased.
> Would this help?
> How do you compare your current GNOME version with the version from
> one year ago?
Like it.
> >> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
> >
> > With what purpose is this asked? No support is given on GNOME Bugzilla.
> > Only minimal on gnome-list.
>
> It's important to have a two-way communication with the users, don't
> you think? So it's important to figure out what channels they actually
> use. In fact, I was thinking to add another question asking if they
> feel there's lack of communication with the team.
I don't see giving support as:
* something which shouldn't be done
* two-way communication
* a way that contributors communicate amongst each other
However, we don't really give support. It would be nice to do more,
but.. it is not done that much (bit on IRC, mailing lists, some forums).
Though I'll say beforehand that I see support as separate from a
developer task.
> Resolving bugs is also support. But feel free to rephrase the question
> to find out how users provide any kind feedback; questions, bug
> reports, issues, etc.
I again disagree. A support issue can be caused by a bug. Such a bug can
be a requirement to solve the support issue. But this doesn't mean
fixing bugs is support.
I still do not see the relevance of this question. Perhaps we could ask
where they learned of the survey?
> >> === 08. Do you have any comments or suggestions for the GNOME team? ===
> >
> > How will you handle all the various comments you'll get? Git received
> > 9000 answers.. who is going to read them all and summarize?
>
> I would, if nobody takes the task. Most likely I will miss a lot of
> important stuff if I'm the only one. But it is important to ask this,
> as the survey is bound to be imperfect, and this is the only place
> where people can actually suggest what to add to the next survey
> (indirectly).
Nice.
Note: I do like the question, my only interest is that we don't ask
something, then ignore the answers.
> > Things I'd like to know:
> > - Multiple choice options to understand the responders technical
> > skills. So if they've submitted a patch, contributed code somewhere,
> > paid to write software, maintain some open source code, etc.
> > That is IMO a nicer way to understand their skills can someone to
> > rate themselves.
>
> How about Zeeshan's suggestion of asking how often they use the terminal?
>
> > - If they use KDE/XFCE/something
>
> That's confusing. You mean if they switch between GNOME and those? Or
> that they used to use GNOME, and now they use that one?
My thought was if they use multiple at the same time.
> How about:
> Which other desktop environments have you used?
I like that. Maybe add a timeframe? E.g. "used in the last three years"?
> > - other things to later on be able to determine if the survey is biased
> > in some way. Don't have concrete ideas atm.
>
> Well, how are you going to determine if the survey is biased in some
> way, or it's the actual GNOME users that are biased in some way?
> Either way, I think the only important bias is geekness.
Don't know, that is why I said I don't have concrete ideas.
--
Regards,
Olav
From: Felipe Contreras < felipe contreras gmail com>
To: Felipe Contreras < felipe contreras gmail com>, desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 11:00:31 +0300
On Mon, Aug 1, 2011 at 12:49 AM, Olav Vitters < olav vitters nl> wrote:
> On Mon, Aug 01, 2011 at 12:16:54AM +0300, Felipe Contreras wrote:
>> On Sun, Jul 31, 2011 at 11:14 PM, Olav Vitters < olav vitters nl> wrote:
>> > On Sun, Jul 31, 2011 at 07:11:34PM +0300, Felipe Contreras wrote:
>> >> Many of these are borrowed from the Git user survey. The results as
>> >> you can see, can be quite interesting:
>> >> https://git.wiki.kernel.org/index.php/GitSurvey2010
>> >>
>> >> It would be great if some sort of notification would popup directly on
>> >> user's desktops, this way it can ensured that the maximum amount of
>> >> people are notified. Otherwise, I think planet GNOME, reddit, twitter,
>> >
>> > That's impossible.
>>
>> Everything is possible.
>>
>> >> Google+ and so on should give plenty of feedback. Maybe also contact
>> >> Ars Technica, LWN, Phornix, and so on would help.
>> >
>> > Those are only technical sites. I think the results are going to be
>> > biased whatever you do.
>>
>> How do you suggest to reach the end users if you already dismissed a
>> pop up directly from the desktop? If there isn't any way, at least
>> it's better than nothing.
>
> I didn't dismiss. I said it was impossible to popup a notification.
I already explained how it would be possible; you create a new package
that checks for notifications on a server. And this new packages is
added as a dependency on distro packages. Perfectly possible.
>> >> === 03. How do you describe the amount of configurations available? ===
>> >
>> > I don't see the relevance of asking this. Furthermore the question is
>> > suggestive. Seems more to prove a point than anything else.
>>
>> I do see the relevance, as I think it has been a big point of
>> contention raised by many users.
>
> Something should be done with a survey. No matter the outcome of this
> question, you won't be able to take these results and change things.
>
> Asking if people want more configuration options goes against why
> options are removed. Ideally everything should happen automatically.
>
> I'm only interested in the cases where it doesn't work.
I other words, you are saying that it doesn't matter if 100% of the
responders of this survey say GNOME has too few options, nothing would
be done? Is there *any* kind of evidence that would convince GNOME ppl
that users want more options? Or is it what the wishes of users are
completely irrelevant?
>> If you don't think it's very important, it could go to the end.
>> Besides, if the results are overwhelmingly pointing to "Too few"?
>> Don't you think it would be worth investigating? If the results end up
>> being "Just enough", then no harm done.
>
> The question is suggestive so the results will be biased. It is better
> to ask something like "does GNOME do what you want?" and then have a
> text field where they can specify what GNOME lacks.
Something like:
=== . In your opinion, which areas in GNOME need improvement? ===
(matrix)
Columns: don't need / a little / some / much
+ applications
+ usability
+ documentation
+ configuration
+ localization (translation)
>> >> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
>> >
>> > With what purpose is this asked? No support is given on GNOME Bugzilla.
>> > Only minimal on gnome-list.
>>
>> It's important to have a two-way communication with the users, don't
>> you think? So it's important to figure out what channels they actually
>> use. In fact, I was thinking to add another question asking if they
>> feel there's lack of communication with the team.
>
> I don't see giving support as:
> * something which shouldn't be done
> * two-way communication
> * a way that contributors communicate amongst each other
>
> However, we don't really give support. It would be nice to do more,
> but.. it is not done that much (bit on IRC, mailing lists, some forums).
> Though I'll say beforehand that I see support as separate from a
> developer task.
I still don't see any suggestions.
>> Resolving bugs is also support. But feel free to rephrase the question
>> to find out how users provide any kind feedback; questions, bug
>> reports, issues, etc.
>
> I again disagree. A support issue can be caused by a bug. Such a bug can
> be a requirement to solve the support issue. But this doesn't mean
> fixing bugs is support.
It definitely is. When a product ends its support, it means bugs are
not going to be fixed any more.
It is important to know how many people actually use bugzilla, and how
many people ask questions in IRC channels, mailing lists, etc.
Maybe:
What channel(s) do you use to communicate with the GNOME team (if any)?
>> > Things I'd like to know:
>> > - Multiple choice options to understand the responders technical
>> > skills. So if they've submitted a patch, contributed code somewhere,
>> > paid to write software, maintain some open source code, etc.
>> > That is IMO a nicer way to understand their skills can someone to
>> > rate themselves.
>>
>> How about Zeeshan's suggestion of asking how often they use the terminal?
>>
>> > - If they use KDE/XFCE/something
>>
>> That's confusing. You mean if they switch between GNOME and those? Or
>> that they used to use GNOME, and now they use that one?
>
> My thought was if they use multiple at the same time.
>
>> How about:
>> Which other desktop environments have you used?
>
> I like that. Maybe add a timeframe? E.g. "used in the last three years"?
Ok.
--
Felipe Contreras
From: Germán Póo-Caamaño < gpoo gnome org>
To: Felipe Contreras < felipe contreras gmail com>
Cc: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 01 Aug 2011 02:00:08 -0700
On Mon, 2011-08-01 at 11:00 +0300, Felipe Contreras wrote:
> On Mon, Aug 1, 2011 at 12:49 AM, Olav Vitters < olav vitters nl> wrote:
> > On Mon, Aug 01, 2011 at 12:16:54AM +0300, Felipe Contreras wrote:
> >> On Sun, Jul 31, 2011 at 11:14 PM, Olav Vitters < olav vitters nl> wrote:
> >> > On Sun, Jul 31, 2011 at 07:11:34PM +0300, Felipe Contreras wrote:
> [...]
> >> >> === 03. How do you describe the amount of configurations available? ===
> >> >
> >> > I don't see the relevance of asking this. Furthermore the question is
> >> > suggestive. Seems more to prove a point than anything else.
> >>
> >> I do see the relevance, as I think it has been a big point of
> >> contention raised by many users.
> >
> > Something should be done with a survey. No matter the outcome of this
> > question, you won't be able to take these results and change things.
> >
> > Asking if people want more configuration options goes against why
> > options are removed. Ideally everything should happen automatically.
> >
> > I'm only interested in the cases where it doesn't work.
>
> I other words, you are saying that it doesn't matter if 100% of the
> responders of this survey say GNOME has too few options, nothing would
> be done? Is there *any* kind of evidence that would convince GNOME ppl
> that users want more options? Or is it what the wishes of users are
> completely irrelevant?
First at all, you need to define a goal, what are you going to do with
the results and what kind of actions would be needed to improve the
results in a future survey.
That said, if you get:
40% users answered 'Too many options'
10% users answered 'just enough'
50% users answered 'few options'
Then, so what? There is no useful information you can get from this.
"What do we need to improve? Add more options." (!?)
Having configuration options is an implementation detail. Olav points
it out correctly, and his suggestions goes in the right directions:
"does GNOME do what you want?" with a text field to specify what it
lacks. The results should be far more concrete than asking whether they
like it or not.
So, how can you formulate better questions for a survey? Taking a text
book of HCI. For instance: http://hcibib.org/tcuid/index.html In
particular the chapters 4 and 5 (Evaluating the Design with and without
users). The type of questions is more or less similar.
Another text book could be http://goo.gl/wkBje (not available online,
though). Part 4 (chapter 20 to 27).
Later, you might want to run an Heuristic analysis in order to get more
concrete and objetive points, set goals, etc. Having several evaluators
will help you to get the common findings and discuss the differences,
which would lead the set the proper questions for a survey or user
study.
Regards,
--
Germán Póo-Caamaño
http://people.gnome.org/~gpoo/
From: Felipe Contreras < felipe contreras gmail com>
To: Germán Póo-Caamaño < gpoo gnome org>
Cc: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 12:21:43 +0300
On Mon, Aug 1, 2011 at 12:00 PM, Germán Póo-Caamaño < gpoo gnome org> wrote:
> On Mon, 2011-08-01 at 11:00 +0300, Felipe Contreras wrote:
>> On Mon, Aug 1, 2011 at 12:49 AM, Olav Vitters < olav vitters nl> wrote:
>> > On Mon, Aug 01, 2011 at 12:16:54AM +0300, Felipe Contreras wrote:
>> >> On Sun, Jul 31, 2011 at 11:14 PM, Olav Vitters < olav vitters nl> wrote:
>> >> > On Sun, Jul 31, 2011 at 07:11:34PM +0300, Felipe Contreras wrote:
>> [...]
>> >> >> === 03. How do you describe the amount of configurations available? ===
>> >> >
>> >> > I don't see the relevance of asking this. Furthermore the question is
>> >> > suggestive. Seems more to prove a point than anything else.
>> >>
>> >> I do see the relevance, as I think it has been a big point of
>> >> contention raised by many users.
>> >
>> > Something should be done with a survey. No matter the outcome of this
>> > question, you won't be able to take these results and change things.
>> >
>> > Asking if people want more configuration options goes against why
>> > options are removed. Ideally everything should happen automatically.
>> >
>> > I'm only interested in the cases where it doesn't work.
>>
>> I other words, you are saying that it doesn't matter if 100% of the
>> responders of this survey say GNOME has too few options, nothing would
>> be done? Is there *any* kind of evidence that would convince GNOME ppl
>> that users want more options? Or is it what the wishes of users are
>> completely irrelevant?
>
> First at all, you need to define a goal, what are you going to do with
> the results and what kind of actions would be needed to improve the
> results in a future survey.
No, you don't.
Having been involved in the development of the Git user survey for
years, I can tell you that there's no concrete goal, it's just to
collect information. Many question didn't seem like they would get
interesting results, but after getting the results, it turned out that
they did. But it's only _afterwards_ that you can say for sure. Sure,
some of the questions look like could be removed, since they are not
providing any value, but it's only _afterwards_ that we know that.
> That said, if you get:
> 40% users answered 'Too many options'
> 10% users answered 'just enough'
> 50% users answered 'few options'
But what if you get:
2% users answered 'Too many options'
10% users answered 'just enough'
88% users answered 'few options'
I repeat, the worst that could happen is that the results of the
question don't provide any value, so you wasted one question... big
deal. You remove it in the next one.
--
Felipe Contreras
From: Shaun McCance < shaunm gnome org>
To: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 01 Aug 2011 09:24:35 -0400
On Mon, 2011-08-01 at 12:21 +0300, Felipe Contreras wrote:
> But what if you get:
> 2% users answered 'Too many options'
> 10% users answered 'just enough'
> 88% users answered 'few options'
>
> I repeat, the worst that could happen is that the results of the
> question don't provide any value, so you wasted one question... big
> deal. You remove it in the next one.
That is still not useful information. Developers aren't going
to add options for the sake of adding options. "Users want
more options? I guess I better hunt through my program to see
what I can make configurable." That's absurd.
We need to know what users want to change, and (importantly)
why they want to change it. Aggregate statistics on this, even
if accurate and significant, are not actionable.
--
Shaun
From: Felipe Contreras < felipe contreras gmail com>
To: Shaun McCance < shaunm gnome org>
Cc: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 16:33:40 +0300
On Mon, Aug 1, 2011 at 4:24 PM, Shaun McCance < shaunm gnome org> wrote:
> On Mon, 2011-08-01 at 12:21 +0300, Felipe Contreras wrote:
>> But what if you get:
>> 2% users answered 'Too many options'
>> 10% users answered 'just enough'
>> 88% users answered 'few options'
>>
>> I repeat, the worst that could happen is that the results of the
>> question don't provide any value, so you wasted one question... big
>> deal. You remove it in the next one.
>
> That is still not useful information. Developers aren't going
> to add options for the sake of adding options. "Users want
> more options? I guess I better hunt through my program to see
> what I can make configurable." That's absurd.
>
> We need to know what users want to change, and (importantly)
> why they want to change it. Aggregate statistics on this, even
> if accurate and significant, are not actionable.
Oh, so you agree that a lot of options are missing? Good, so what are
you doing to identify those options? Do the majority of GNOME
developers agree that there are too few options?
The purpose of this survey is not to identify those options, you need
other tools for that. Anyway, if GNOME developers agree that there are
too few options, this question doesn't hurt, and if they don't, this
might help to convince them.
--
Felipe Contreras
From: Pascal Terjan < pterjan gmail com>
To: Felipe Contreras < felipe contreras gmail com>
Cc: Shaun McCance < shaunm gnome org>, desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 14:41:26 +0100
On Mon, Aug 1, 2011 at 14:33, Felipe Contreras
< felipe contreras gmail com> wrote:
> On Mon, Aug 1, 2011 at 4:24 PM, Shaun McCance < shaunm gnome org> wrote:
>> On Mon, 2011-08-01 at 12:21 +0300, Felipe Contreras wrote:
>>> But what if you get:
>>> 2% users answered 'Too many options'
>>> 10% users answered 'just enough'
>>> 88% users answered 'few options'
>>>
>>> I repeat, the worst that could happen is that the results of the
>>> question don't provide any value, so you wasted one question... big
>>> deal. You remove it in the next one.
>>
>> That is still not useful information. Developers aren't going
>> to add options for the sake of adding options. "Users want
>> more options? I guess I better hunt through my program to see
>> what I can make configurable." That's absurd.
>>
>> We need to know what users want to change, and (importantly)
>> why they want to change it. Aggregate statistics on this, even
>> if accurate and significant, are not actionable.
>
> Oh, so you agree that a lot of options are missing? Good, so what are
> you doing to identify those options? Do the majority of GNOME
> developers agree that there are too few options?
I don't see this in his email, try to read again what you quoted.
He agrees that some things may be missing for some users.
You transform this into "a lot of options are missing". This is not
what he says.
> The purpose of this survey is not to identify those options, you need
> other tools for that. Anyway, if GNOME developers agree that there are
> too few options, this question doesn't hurt, and if they don't, this
> might help to convince them.
From: Alan Cox < alan lxorguk ukuu org uk>
To: Pascal Terjan < pterjan gmail com>
Cc: Shaun McCance < shaunm gnome org>, desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 14:52:52 +0100
What are you actually trying to understand ?
If there are tradeoffs (and consensus that the tradeoff is real) then you
can ask questions where you rate relative importance of features in
various combinations It's often used to weight things like
price/screensize, screensize/battery life, battery life/weight and so on.
Is there a reason it can't be used to ask about ease of
configuration/flexibility, flexibility/reliability, etc
I agree asking "are there enough options" isn't useful but you could ask
"If you could configure three extra things what would they be"
and leave a text box. If most people leave it blank it tells you a lot,
if everyone lists a thing you can do it tells you about doc problems, if
there is a consensus on some things then it tells you other stuff.
Alan
From: Shaun McCance < shaunm gnome org>
To: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 01 Aug 2011 10:02:33 -0400
On Mon, 2011-08-01 at 16:33 +0300, Felipe Contreras wrote:
> On Mon, Aug 1, 2011 at 4:24 PM, Shaun McCance < shaunm gnome org> wrote:
> > On Mon, 2011-08-01 at 12:21 +0300, Felipe Contreras wrote:
> >> But what if you get:
> >> 2% users answered 'Too many options'
> >> 10% users answered 'just enough'
> >> 88% users answered 'few options'
> >>
> >> I repeat, the worst that could happen is that the results of the
> >> question don't provide any value, so you wasted one question... big
> >> deal. You remove it in the next one.
> >
> > That is still not useful information. Developers aren't going
> > to add options for the sake of adding options. "Users want
> > more options? I guess I better hunt through my program to see
> > what I can make configurable." That's absurd.
> >
> > We need to know what users want to change, and (importantly)
> > why they want to change it. Aggregate statistics on this, even
> > if accurate and significant, are not actionable.
>
> Oh, so you agree that a lot of options are missing?
I didn't say that. And that statement illustrates to me that you
have a very strong agenda. I don't think somebody with an agenda
can design an unbiased survey.
> Good, so what are you doing to identify those options?
As a member of the documentation team, I regularly run ad-hoc
user tests to determine (1) what aspects of our software need
better explaining, and (2) how well our help actually helps
people. I regularly discuss findings with the design team,
and I file bugs.
--
Shaun
From: Felipe Contreras < felipe contreras gmail com>
To: Shaun McCance < shaunm gnome org>
Cc: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 17:32:02 +0300
On Mon, Aug 1, 2011 at 5:02 PM, Shaun McCance < shaunm gnome org> wrote:
> On Mon, 2011-08-01 at 16:33 +0300, Felipe Contreras wrote:
>> On Mon, Aug 1, 2011 at 4:24 PM, Shaun McCance < shaunm gnome org> wrote:
>> > On Mon, 2011-08-01 at 12:21 +0300, Felipe Contreras wrote:
>> >> But what if you get:
>> >> 2% users answered 'Too many options'
>> >> 10% users answered 'just enough'
>> >> 88% users answered 'few options'
>> >>
>> >> I repeat, the worst that could happen is that the results of the
>> >> question don't provide any value, so you wasted one question... big
>> >> deal. You remove it in the next one.
>> >
>> > That is still not useful information. Developers aren't going
>> > to add options for the sake of adding options. "Users want
>> > more options? I guess I better hunt through my program to see
>> > what I can make configurable." That's absurd.
>> >
>> > We need to know what users want to change, and (importantly)
>> > why they want to change it. Aggregate statistics on this, even
>> > if accurate and significant, are not actionable.
>>
>> Oh, so you agree that a lot of options are missing?
>
> I didn't say that. And that statement illustrates to me that you
> have a very strong agenda. I don't think somebody with an agenda
> can design an unbiased survey.
If you don't agree that options are missing, then what's the point of
skipping the question, and going directly to ask what they would
change? (which I don't see how it can be done in this survey anyway).
First I would like to try to identify the need for change, if any.
*I* am not designing the survey, *we* are.
--
Felipe Contreras
From: Shaun McCance < shaunm gnome org>
To: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 01 Aug 2011 10:51:56 -0400
On Mon, 2011-08-01 at 17:32 +0300, Felipe Contreras wrote:
> On Mon, Aug 1, 2011 at 5:02 PM, Shaun McCance < shaunm gnome org> wrote:
> > On Mon, 2011-08-01 at 16:33 +0300, Felipe Contreras wrote:
> >> On Mon, Aug 1, 2011 at 4:24 PM, Shaun McCance < shaunm gnome org> wrote:
> >> > On Mon, 2011-08-01 at 12:21 +0300, Felipe Contreras wrote:
> >> >> But what if you get:
> >> >> 2% users answered 'Too many options'
> >> >> 10% users answered 'just enough'
> >> >> 88% users answered 'few options'
> >> >>
> >> >> I repeat, the worst that could happen is that the results of the
> >> >> question don't provide any value, so you wasted one question... big
> >> >> deal. You remove it in the next one.
> >> >
> >> > That is still not useful information. Developers aren't going
> >> > to add options for the sake of adding options. "Users want
> >> > more options? I guess I better hunt through my program to see
> >> > what I can make configurable." That's absurd.
> >> >
> >> > We need to know what users want to change, and (importantly)
> >> > why they want to change it. Aggregate statistics on this, even
> >> > if accurate and significant, are not actionable.
> >>
> >> Oh, so you agree that a lot of options are missing?
> >
> > I didn't say that. And that statement illustrates to me that you
> > have a very strong agenda. I don't think somebody with an agenda
> > can design an unbiased survey.
>
> If you don't agree that options are missing, then what's the point of
> skipping the question, and going directly to ask what they would
> change? (which I don't see how it can be done in this survey anyway).
> First I would like to try to identify the need for change, if any.
Sigh. I am open to the idea that there are aspects of our software
which can be improved, and that in some cases, improving it means
adding an option. This is a far cry from agreeing that a lot of
options are missing.
You seem to want to establish that we have too few options, so we
can go look for which options we should add. That's backwards. We
should instead look for ways to improve the software, and if that
happens to involve adding an option, add it.
Options are a means to an end. They should never be the goal.
--
Shaun
From: Olav Vitters < olav vitters nl>
To: Felipe Contreras < felipe contreras gmail com>
Cc: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 11:05:07 +0200
On Mon, Aug 01, 2011 at 11:00:31AM +0300, Felipe Contreras wrote:
> On Mon, Aug 1, 2011 at 12:49 AM, Olav Vitters < olav vitters nl> wrote:
> > On Mon, Aug 01, 2011 at 12:16:54AM +0300, Felipe Contreras wrote:
> >> On Sun, Jul 31, 2011 at 11:14 PM, Olav Vitters < olav vitters nl> wrote:
> >> > On Sun, Jul 31, 2011 at 07:11:34PM +0300, Felipe Contreras wrote:
> >> >> It would be great if some sort of notification would popup directly on
> >> >> user's desktops, this way it can ensured that the maximum amount of
> >> >> people are notified. Otherwise, I think planet GNOME, reddit, twitter,
> >> >
> >> > That's impossible.
> >>
> >> Everything is possible.
> >>
> >> >> Google+ and so on should give plenty of feedback. Maybe also contact
> >> >> Ars Technica, LWN, Phornix, and so on would help.
> >> >
> >> > Those are only technical sites. I think the results are going to be
> >> > biased whatever you do.
> >>
> >> How do you suggest to reach the end users if you already dismissed a
> >> pop up directly from the desktop? If there isn't any way, at least
> >> it's better than nothing.
> >
> > I didn't dismiss. I said it was impossible to popup a notification.
>
> I already explained how it would be possible; you create a new package
> that checks for notifications on a server. And this new packages is
> added as a dependency on distro packages. Perfectly possible.
I guess you're talking about popping up a notification in future GNOME
versions? I was talking about existing desktops as various things in
your initial email indicated you wanted to ask existing people (the
GNOME version question + 2011 in the subject).
Note: I didn't see that explanation. Please just copy/paste when it is
clear I am not aware.
> >> >> === 03. How do you describe the amount of configurations available? ===
> >> >
> >> > I don't see the relevance of asking this. Furthermore the question is
> >> > suggestive. Seems more to prove a point than anything else.
> >>
> >> I do see the relevance, as I think it has been a big point of
> >> contention raised by many users.
> >
> > Something should be done with a survey. No matter the outcome of this
> > question, you won't be able to take these results and change things.
> >
> > Asking if people want more configuration options goes against why
> > options are removed. Ideally everything should happen automatically.
> >
> > I'm only interested in the cases where it doesn't work.
>
> I other words, you are saying that it doesn't matter if 100% of the
> responders of this survey say GNOME has too few options, nothing would
> be done? Is there *any* kind of evidence that would convince GNOME ppl
> that users want more options? Or is it what the wishes of users are
> completely irrelevant?
That doesn't reflect what I said. As such, the statement that nothing
would be done is not accurate. What I'd rather have is something like
"Does GNOME do everything what you want to do?"
and then
"not at all", "somewhat", "mostly", "everything"
then maybe a small textbox to indicate what it doesn't do... but that
would have to be analyzed (summarized) again.
> >> If you don't think it's very important, it could go to the end.
> >> Besides, if the results are overwhelmingly pointing to "Too few"?
> >> Don't you think it would be worth investigating? If the results end up
> >> being "Just enough", then no harm done.
> >
> > The question is suggestive so the results will be biased. It is better
> > to ask something like "does GNOME do what you want?" and then have a
> > text field where they can specify what GNOME lacks.
>
> Something like:
>
> === . In your opinion, which areas in GNOME need improvement? ===
> (matrix)
>
> Columns: don't need / a little / some / much
> + applications
> + usability
> + documentation
> + configuration
> + localization (translation)
The question already suggests that GNOME needs improvement. I rather
have non-suggestive questions. Further, I think 'applications' is vague.
I don't think many people will know which parts are GNOME and which
parts are not.
e.g. 'How happy are you with GNOME in regards to'
(happy is not as suggestive as you might think; though another word
might be better)
and then rephrase some of the items (usability -> ease of use;
localization -> available in your language; etc)
if we then get responses indicating 'very unhappy' then we know what to
work upon. Perhaps also something which indicates stability.. but then
explained in a simple way ('crashers' => bad).
> >> >> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
> >> >
> >> > With what purpose is this asked? No support is given on GNOME Bugzilla.
> >> > Only minimal on gnome-list.
> >>
> >> It's important to have a two-way communication with the users, don't
> >> you think? So it's important to figure out what channels they actually
> >> use. In fact, I was thinking to add another question asking if they
> >> feel there's lack of communication with the team.
> >
> > I don't see giving support as:
> > * something which shouldn't be done
> > * two-way communication
> > * a way that contributors communicate amongst each other
> >
> > However, we don't really give support. It would be nice to do more,
> > but.. it is not done that much (bit on IRC, mailing lists, some forums).
> > Though I'll say beforehand that I see support as separate from a
> > developer task.
>
> I still don't see any suggestions.
My suggestion is to remove the question.
> >> Resolving bugs is also support. But feel free to rephrase the question
> >> to find out how users provide any kind feedback; questions, bug
> >> reports, issues, etc.
> >
> > I again disagree. A support issue can be caused by a bug. Such a bug can
> > be a requirement to solve the support issue. But this doesn't mean
> > fixing bugs is support.
>
> It definitely is. When a product ends its support, it means bugs are
> not going to be fixed any more.
I already explained that support can mean bugs, but bugs doesn't mean
support. You're repeating what I said without addressing my suggestion.
Meaning:
S->B
does not imply
B->S
(bugs, support)
We do not really give support. However, a small amount of support is
given. E.g. in #gnome channel. Questions regarding how to use an applet,
evolution, set keyboard shortcuts and whatnot. No bugs (commits) for
GNOME 2.30 will be fixed anymore (though in practice they'll still be
fixed in the latest version), but that doesn't mean I won't answer
questions in #gnome anymore.
> It is important to know how many people actually use bugzilla, and how
> many people ask questions in IRC channels, mailing lists, etc.
With what purpose? What would GNOME get out of this information?
> Maybe:
> What channel(s) do you use to communicate with the GNOME team (if any)?
This doesn't reflect support anymore?
> >> > Things I'd like to know:
Other idea is asking which distribution they use. Might be handy as I
saw someone mention GNOME 3 is very crash prone and it should not be. We
might also get people who might think 'Unity' and 'KDE' == GNOME.
Perhaps ask what their desktop looks similar to with some example
images.
--
Regards,
Olav
From: Felipe Contreras < felipe contreras gmail com>
To: Felipe Contreras < felipe contreras gmail com>, desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 12:45:11 +0300
On Mon, Aug 1, 2011 at 12:05 PM, Olav Vitters < olav vitters nl> wrote:
> On Mon, Aug 01, 2011 at 11:00:31AM +0300, Felipe Contreras wrote:
>> On Mon, Aug 1, 2011 at 12:49 AM, Olav Vitters < olav vitters nl> wrote:
>> > On Mon, Aug 01, 2011 at 12:16:54AM +0300, Felipe Contreras wrote:
>> >> On Sun, Jul 31, 2011 at 11:14 PM, Olav Vitters < olav vitters nl> wrote:
>> >> > On Sun, Jul 31, 2011 at 07:11:34PM +0300, Felipe Contreras wrote:
>> >> >> It would be great if some sort of notification would popup directly on
>> >> >> user's desktops, this way it can ensured that the maximum amount of
>> >> >> people are notified. Otherwise, I think planet GNOME, reddit, twitter,
>> >> >
>> >> > That's impossible.
>> >>
>> >> Everything is possible.
>> >>
>> >> >> Google+ and so on should give plenty of feedback. Maybe also contact
>> >> >> Ars Technica, LWN, Phornix, and so on would help.
>> >> >
>> >> > Those are only technical sites. I think the results are going to be
>> >> > biased whatever you do.
>> >>
>> >> How do you suggest to reach the end users if you already dismissed a
>> >> pop up directly from the desktop? If there isn't any way, at least
>> >> it's better than nothing.
>> >
>> > I didn't dismiss. I said it was impossible to popup a notification.
>>
>> I already explained how it would be possible; you create a new package
>> that checks for notifications on a server. And this new packages is
>> added as a dependency on distro packages. Perfectly possible.
>
> I guess you're talking about popping up a notification in future GNOME
> versions? I was talking about existing desktops as various things in
> your initial email indicated you wanted to ask existing people (the
> GNOME version question + 2011 in the subject).
>
> Note: I didn't see that explanation. Please just copy/paste when it is
> clear I am not aware.
---
My idea was to have a GNOME notifications package, or something, that
would connect to a site, and then fetch any relevant notifications.
The notification in this case would be a link to the online
user-survey, and nothing else.
Of course, the user should have the option to disable such
notifications (perhaps even from the notification itself), but given
than they would probably happen only a few times per year, I don't
think there such a big hurry to have that option (if needed at all).
---
>> >> >> === 03. How do you describe the amount of configurations available? ===
>> >> >
>> >> > I don't see the relevance of asking this. Furthermore the question is
>> >> > suggestive. Seems more to prove a point than anything else.
>> >>
>> >> I do see the relevance, as I think it has been a big point of
>> >> contention raised by many users.
>> >
>> > Something should be done with a survey. No matter the outcome of this
>> > question, you won't be able to take these results and change things.
>> >
>> > Asking if people want more configuration options goes against why
>> > options are removed. Ideally everything should happen automatically.
>> >
>> > I'm only interested in the cases where it doesn't work.
>>
>> I other words, you are saying that it doesn't matter if 100% of the
>> responders of this survey say GNOME has too few options, nothing would
>> be done? Is there *any* kind of evidence that would convince GNOME ppl
>> that users want more options? Or is it what the wishes of users are
>> completely irrelevant?
>
> That doesn't reflect what I said. As such, the statement that nothing
> would be done is not accurate. What I'd rather have is something like
> "Does GNOME do everything what you want to do?"
> and then
> "not at all", "somewhat", "mostly", "everything"
>
> then maybe a small textbox to indicate what it doesn't do... but that
> would have to be analyzed (summarized) again.
Yes, but if a significant percentage of the people answer "not at
all", or "somewhat", then the survey would be a bit wasted, since now
you have to wait another year to ask in more detail. You could also go
to the free text and try to make sense of it, but that would be too
much work to plot in any sensible way.
>> >> If you don't think it's very important, it could go to the end.
>> >> Besides, if the results are overwhelmingly pointing to "Too few"?
>> >> Don't you think it would be worth investigating? If the results end up
>> >> being "Just enough", then no harm done.
>> >
>> > The question is suggestive so the results will be biased. It is better
>> > to ask something like "does GNOME do what you want?" and then have a
>> > text field where they can specify what GNOME lacks.
>>
>> Something like:
>>
>> === . In your opinion, which areas in GNOME need improvement? ===
>> (matrix)
>>
>> Columns: don't need / a little / some / much
>> + applications
>> + usability
>> + documentation
>> + configuration
>> + localization (translation)
>
> The question already suggests that GNOME needs improvement.
Are you kidding me? Nothing is perfect, everything needs improvement,
everyone knows that.
However, people do pick the option "don't care" and "don't need" in
Git's survey, if that's truly the case:
https://git.wiki.kernel.org/index.php/GitSurvey2010#20.
_In_your_opinion.2C_which_areas_in_Git_need_improvement.3F
> I rather have non-suggestive questions.
The purpose of the question is not figure out if GNOME is perfect or
not. But to find out if the user could vote for what to improve in
GNOME, what would that be. IMO the worst would be if most users select
"don't care" in all of them, because then you would have no guidance
at all to what should be done. Unless of course you are not looking
for areas of improvement.
> Further, I think 'applications' is vague.
> I don't think many people will know which parts are GNOME and which
> parts are not.
>
> e.g. 'How happy are you with GNOME in regards to'
>
> (happy is not as suggestive as you might think; though another word
> might be better)
I do think it's suggestive. I also think improvement is suggestive,
but I think the suggestion that GNOME needs improvement is a good one,
specially if you are looking for areas of improvement.
>> >> >> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
>> >> >
>> >> > With what purpose is this asked? No support is given on GNOME Bugzilla.
>> >> > Only minimal on gnome-list.
>> >>
>> >> It's important to have a two-way communication with the users, don't
>> >> you think? So it's important to figure out what channels they actually
>> >> use. In fact, I was thinking to add another question asking if they
>> >> feel there's lack of communication with the team.
>> >
>> > I don't see giving support as:
>> > * something which shouldn't be done
>> > * two-way communication
>> > * a way that contributors communicate amongst each other
>> >
>> > However, we don't really give support. It would be nice to do more,
>> > but.. it is not done that much (bit on IRC, mailing lists, some forums).
>> > Though I'll say beforehand that I see support as separate from a
>> > developer task.
>>
>> I still don't see any suggestions.
>
> My suggestion is to remove the question.
If people can respond in other ways: configuration => much improvement
needed. Then I'm fine with that.
>> It is important to know how many people actually use bugzilla, and how
>> many people ask questions in IRC channels, mailing lists, etc.
>
> With what purpose? What would GNOME get out of this information?
Suppose 90% of responders don't use any channel of communication.
Perhaps you would then arrive to the conclusion that more channels of
communications are needed, or the current ones need to be advertised
better.
>> Maybe:
>> What channel(s) do you use to communicate with the GNOME team (if any)?
>
> This doesn't reflect support anymore?
That is irrelevant. Why are you pushing so hard against this? Are you
trying to say that if users don't have any means of giving feedback to
the GNOME team, that is _good_?
>> >> > Things I'd like to know:
>
> Other idea is asking which distribution they use. Might be handy as I
> saw someone mention GNOME 3 is very crash prone and it should not be. We
> might also get people who might think 'Unity' and 'KDE' == GNOME.
> Perhaps ask what their desktop looks similar to with some example
> images.
I think if they don't even know what GNOME is, the answers will be
messy. Like GNOME version 4.7. But sure, a picture to select might
help, although that doesn't seem to be available in questionpro.com.
--
Felipe Contreras
From: Olav Vitters < olav vitters nl>
To: Felipe Contreras < felipe contreras gmail com>
Cc: desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 13:37:42 +0200
(snipping a lot)
On Mon, Aug 01, 2011 at 12:45:11PM +0300, Felipe Contreras wrote:
> On Mon, Aug 1, 2011 at 12:05 PM, Olav Vitters < olav vitters nl> wrote:
> > On Mon, Aug 01, 2011 at 11:00:31AM +0300, Felipe Contreras wrote:
> >> On Mon, Aug 1, 2011 at 12:49 AM, Olav Vitters < olav vitters nl> wrote:
> > That doesn't reflect what I said. As such, the statement that nothing
> > would be done is not accurate. What I'd rather have is something like
> > "Does GNOME do everything what you want to do?"
> > and then
> > "not at all", "somewhat", "mostly", "everything"
> >
> > then maybe a small textbox to indicate what it doesn't do... but that
> > would have to be analyzed (summarized) again.
>
> Yes, but if a significant percentage of the people answer "not at
> all", or "somewhat", then the survey would be a bit wasted, since now
> you have to wait another year to ask in more detail. You could also go
> to the free text and try to make sense of it, but that would be too
> much work to plot in any sensible way.
I don't see how your suggestion is better? 'Configurability' to me is
meaningless as there is not much what we could do with the outcome.
> >> >> If you don't think it's very important, it could go to the end.
> >> >> Besides, if the results are overwhelmingly pointing to "Too few"?
> >> >> Don't you think it would be worth investigating? If the results end up
> >> >> being "Just enough", then no harm done.
> >> >
> >> > The question is suggestive so the results will be biased. It is better
> >> > to ask something like "does GNOME do what you want?" and then have a
> >> > text field where they can specify what GNOME lacks.
> >>
> >> Something like:
> >>
> >> === . In your opinion, which areas in GNOME need improvement? ===
> >> (matrix)
> >>
> >> Columns: don't need / a little / some / much
> >> + applications
> >> + usability
> >> + documentation
> >> + configuration
> >> + localization (translation)
> >
> > The question already suggests that GNOME needs improvement.
>
> Are you kidding me? Nothing is perfect, everything needs improvement,
> everyone knows that.
I do not agree. Some things are good enough. Maybe not from a developer
standpoint and it is not that I'd not appreciate further development,
but sometimes I am perfectly happy with the current development.
I thought the question had the intend to check if people like the GNOME
version that they are using?
> However, people do pick the option "don't care" and "don't need" in
> Git's survey, if that's truly the case:
> https://git.wiki.kernel.org/index.php/GitSurvey2010#20.
_In_your_opinion.2C_which_areas_in_Git_need_improvement.3F
You're not really addressing my concern which is that the question is
suggestive. Furthermore, this seems to indicate that even with such a
question, some people already indicate that Git is good enough.
> > I rather have non-suggestive questions.
>
> The purpose of the question is not figure out if GNOME is perfect or
> not. But to find out if the user could vote for what to improve in
> GNOME, what would that be. IMO the worst would be if most users select
> "don't care" in all of them, because then you would have no guidance
> at all to what should be done. Unless of course you are not looking
> for areas of improvement.
If people are not happy with certain things, then those things need
improvement. An option might be one of those things, but that is
something which needs to be analyzed, not assumed.
I'd be curious to know what they're happy with as well as what users are
not happy with.
Further, I was not thinking about specifically using a survey as a
method to get improvement ideas. The reply from Germán Póo-Caamaño
also nicely explains other means to get such feedback.
> > Further, I think 'applications' is vague.
> > I don't think many people will know which parts are GNOME and which
> > parts are not.
> >
> > e.g. 'How happy are you with GNOME in regards to'
> >
> > (happy is not as suggestive as you might think; though another word
> > might be better)
>
> I do think it's suggestive. I also think improvement is suggestive,
> but I think the suggestion that GNOME needs improvement is a good one,
> specially if you are looking for areas of improvement.
I rather have a non-suggestive survey so that the outcome will be more
acceptable by the various developers.
Note that I've read various articles written that it is easier for
people to be critical and negative then to praise. Giving praise is
especially difficult once you've been asked to be critical. I've
suggested a positive tone due to that and seeing this effect in
practice.
> >> >> >> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
> >> >> >
> >> >> > With what purpose is this asked? No support is given on GNOME Bugzilla.
> >> >> > Only minimal on gnome-list.
> >> >>
> >> >> It's important to have a two-way communication with the users, don't
> >> >> you think? So it's important to figure out what channels they actually
> >> >> use. In fact, I was thinking to add another question asking if they
> >> >> feel there's lack of communication with the team.
> >> >
> >> > I don't see giving support as:
> >> > * something which shouldn't be done
> >> > * two-way communication
> >> > * a way that contributors communicate amongst each other
> >> >
> >> > However, we don't really give support. It would be nice to do more,
> >> > but.. it is not done that much (bit on IRC, mailing lists, some forums).
> >> > Though I'll say beforehand that I see support as separate from a
> >> > developer task.
> >>
> >> I still don't see any suggestions.
> >
> > My suggestion is to remove the question.
>
> If people can respond in other ways: configuration => much improvement
> needed. Then I'm fine with that.
Configuration is a meaningless term to me. I don't understand why we not
just ask a direct question "does it do what you want?".
Loads of options has various bad effects. Maintenance is more difficult,
it becomes more confusing, the option might be something similar to 'do
not crash' (meaning: it should be solved differently an no option should
be added), etc. See for instance the following blog article which more
or less explains what I mean:
http://headrush.typepad.com/creating_passionate_users/2005/06/featuritis_vs_t.html
> >> It is important to know how many people actually use bugzilla, and how
> >> many people ask questions in IRC channels, mailing lists, etc.
> >
> > With what purpose? What would GNOME get out of this information?
>
> Suppose 90% of responders don't use any channel of communication.
> Perhaps you would then arrive to the conclusion that more channels of
> communications are needed, or the current ones need to be advertised
> better.
But I already said that I think we should have support options and
currently have none? So why ask this?
> >> Maybe:
> >> What channel(s) do you use to communicate with the GNOME team (if any)?
> >
> > This doesn't reflect support anymore?
>
> That is irrelevant. Why are you pushing so hard against this? Are you
> trying to say that if users don't have any means of giving feedback to
> the GNOME team, that is _good_?
I'm not, I'm just checking as I'm getting confused with (in my view)
the various topic changes.
Initially you talk about support, then communication, then feedback.
I've stated that we do not really have any means of support and wondered
why the question is suddenly different.
>From another answer I gather you want some way to have people vote for
improvement ideas which is I guess what you mean with 'feedback' or do
you mean 'survey' as feedback?
Regarding voting: That was discussed on d-d-l a long time ago. In short:
various concerns. Not against improvement, but in practice: We rather
not improve things via internet voting, but more improve things after
investigating what needs improvement based on actually seeing what does
not work and fix it possibly in another way. For that you'd need to
understand why something is asked for.
I'm not against changing things, I am not against a survey.
--
Regards,
Olav
From: Felipe Contreras < felipe contreras gmail com>
To: Felipe Contreras < felipe contreras gmail com>, desktop-devel-list gnome org
Subject: Re: GNOME user survey 2011
Date: Mon, 1 Aug 2011 17:16:21 +0300
On Mon, Aug 1, 2011 at 2:37 PM, Olav Vitters < olav vitters nl> wrote:
> (snipping a lot)
>
> On Mon, Aug 01, 2011 at 12:45:11PM +0300, Felipe Contreras wrote:
>> On Mon, Aug 1, 2011 at 12:05 PM, Olav Vitters < olav vitters nl> wrote:
>> > On Mon, Aug 01, 2011 at 11:00:31AM +0300, Felipe Contreras wrote:
>> >> On Mon, Aug 1, 2011 at 12:49 AM, Olav Vitters < olav vitters nl> wrote:
>> > That doesn't reflect what I said. As such, the statement that nothing
>> > would be done is not accurate. What I'd rather have is something like
>> > "Does GNOME do everything what you want to do?"
>> > and then
>> > "not at all", "somewhat", "mostly", "everything"
>> >
>> > then maybe a small textbox to indicate what it doesn't do... but that
>> > would have to be analyzed (summarized) again.
>>
>> Yes, but if a significant percentage of the people answer "not at
>> all", or "somewhat", then the survey would be a bit wasted, since now
>> you have to wait another year to ask in more detail. You could also go
>> to the free text and try to make sense of it, but that would be too
>> much work to plot in any sensible way.
>
> I don't see how your suggestion is better? 'Configurability' to me is
> meaningless as there is not much what we could do with the outcome.
It's almost as meaningless as "Does GNOME do everything what you want
to do?", except that you would know that it's related to the
configuration, and not usability.
>> >> >> If you don't think it's very important, it could go to the end.
>> >> >> Besides, if the results are overwhelmingly pointing to "Too few"?
>> >> >> Don't you think it would be worth investigating? If the results end up
>> >> >> being "Just enough", then no harm done.
>> >> >
>> >> > The question is suggestive so the results will be biased. It is better
>> >> > to ask something like "does GNOME do what you want?" and then have a
>> >> > text field where they can specify what GNOME lacks.
>> >>
>> >> Something like:
>> >>
>> >> === . In your opinion, which areas in GNOME need improvement? ===
>> >> (matrix)
>> >>
>> >> Columns: don't need / a little / some / much
>> >> + applications
>> >> + usability
>> >> + documentation
>> >> + configuration
>> >> + localization (translation)
>> >
>> > The question already suggests that GNOME needs improvement.
>>
>> Are you kidding me? Nothing is perfect, everything needs improvement,
>> everyone knows that.
>
> I do not agree. Some things are good enough. Maybe not from a developer
> standpoint and it is not that I'd not appreciate further development,
> but sometimes I am perfectly happy with the current development.
Then what's the point of doing any more development? Let's call GNOME
3.0 the pinnacle of desktop environment software and don't release any
more versions.
No, nothing is perfect, everything needs improvements.
> I thought the question had the intend to check if people like the GNOME
> version that they are using?
Yes, but you didn't like the original question, so I changed it to
something similar to what Git does.
>> However, people do pick the option "don't care" and "don't need" in
>> Git's survey, if that's truly the case:
>> https://git.wiki.kernel.org/index.php/GitSurvey2010#20.
_In_your_opinion.2C_which_areas_in_Git_need_improvement.3F
>
> You're not really addressing my concern which is that the question is
> suggestive. Furthermore, this seems to indicate that even with such a
> question, some people already indicate that Git is good enough.
If it is suggesting, why a lot of people answer "don't care" in the
Git survey. Either being suggestive is a problem, or it's not. You are
saying two things at the same time.
>> > I rather have non-suggestive questions.
>>
>> The purpose of the question is not figure out if GNOME is perfect or
>> not. But to find out if the user could vote for what to improve in
>> GNOME, what would that be. IMO the worst would be if most users select
>> "don't care" in all of them, because then you would have no guidance
>> at all to what should be done. Unless of course you are not looking
>> for areas of improvement.
>
> If people are not happy with certain things, then those things need
> improvement. An option might be one of those things, but that is
> something which needs to be analyzed, not assumed.
>
> I'd be curious to know what they're happy with as well as what users are
> not happy with.
Yes, but we haven't found a neutral language. Happy is biased towards
one side, and improvement is biased towards the other. However, I'd
say "improvement" is less biased, because it is implicit in each and
every human endeavor, whereas happiness is not.
> Further, I was not thinking about specifically using a survey as a
> method to get improvement ideas.
What is wrong with getting improvement ideas?
> The reply from Germán Póo-Caamaño
> also nicely explains other means to get such feedback.
I don't know what you are talking about.
>> > Further, I think 'applications' is vague.
>> > I don't think many people will know which parts are GNOME and which
>> > parts are not.
>> >
>> > e.g. 'How happy are you with GNOME in regards to'
>> >
>> > (happy is not as suggestive as you might think; though another word
>> > might be better)
>>
>> I do think it's suggestive. I also think improvement is suggestive,
>> but I think the suggestion that GNOME needs improvement is a good one,
>> specially if you are looking for areas of improvement.
>
> I rather have a non-suggestive survey so that the outcome will be more
> acceptable by the various developers.
Oh, the survey is non-suggestive, it's only this question that we are
debating about the tone, and so far, I haven't heard of a
non-suggestive way to ask this question.
All I know, is that the Git survey has been using "improvement" for
years, and nobody is dismissing the results survey. People think Git
needs improvements... That's no surprise.
> Note that I've read various articles written that it is easier for
> people to be critical and negative then to praise. Giving praise is
> especially difficult once you've been asked to be critical. I've
> suggested a positive tone due to that and seeing this effect in
> practice.
So you would rather get praise than get ideas for improvement? I
prefer "improvement" if there's no way to convince you, then
"happiness" is fine.
>> >> >> >> === 06. What channel(s) do you use to request help about GNOME (if any)? ===
>> >> >> >
>> >> >> > With what purpose is this asked? No support is given on GNOME Bugzilla.
>> >> >> > Only minimal on gnome-list.
>> >> >>
>> >> >> It's important to have a two-way communication with the users, don't
>> >> >> you think? So it's important to figure out what channels they actually
>> >> >> use. In fact, I was thinking to add another question asking if they
>> >> >> feel there's lack of communication with the team.
>> >> >
>> >> > I don't see giving support as:
>> >> > * something which shouldn't be done
>> >> > * two-way communication
>> >> > * a way that contributors communicate amongst each other
>> >> >
>> >> > However, we don't really give support. It would be nice to do more,
>> >> > but.. it is not done that much (bit on IRC, mailing lists, some forums).
>> >> > Though I'll say beforehand that I see support as separate from a
>> >> > developer task.
>> >>
>> >> I still don't see any suggestions.
>> >
>> > My suggestion is to remove the question.
>>
>> If people can respond in other ways: configuration => much improvement
>> needed. Then I'm fine with that.
>
> Configuration is a meaningless term to me. I don't understand why we not
> just ask a direct question "does it do what you want?".
After thinking about it you are right. I think that question is mixing
both usability and configuration, and thus would be harder to
determine what actions to do (if any). But perhaps that's not too bad.
> Loads of options has various bad effects. Maintenance is more difficult,
> it becomes more confusing, the option might be something similar to 'do
> not crash' (meaning: it should be solved differently an no option should
> be added), etc. See for instance the following blog article which more
> or less explains what I mean:
> http://headrush.typepad.com/creating_passionate_users/2005/06/featuritis_vs_t.html
Yes, yes, GNOME developers have thrown that argument ad nauseam. But
it's not a valid reason to don't add features:
1. How do you know you are on the happy user peak without any numbers?
2. Can't this be solved easily by adding an "advanced mode"
If you are in the happy user peak, then the majority of responders
will say "just enough". No?
Moreover, I am perfectly sure there's way to achieve configurability,
*and* keep the code maintenable, but it requires more work. You don't
have to aim for that kind of quality, but at least you would know
where your users stand.
>> >> It is important to know how many people actually use bugzilla, and how
>> >> many people ask questions in IRC channels, mailing lists, etc.
>> >
>> > With what purpose? What would GNOME get out of this information?
>>
>> Suppose 90% of responders don't use any channel of communication.
>> Perhaps you would then arrive to the conclusion that more channels of
>> communications are needed, or the current ones need to be advertised
>> better.
>
> But I already said that I think we should have support options and
> currently have none? So why ask this?
To see how the usage of different channels is distributed? Say
StackOverflow turned out very high for Git, if something similar
happens for GNOME, maybe some GNOME developers would hang around there
more.
>> >> Maybe:
>> >> What channel(s) do you use to communicate with the GNOME team (if any)?
>> >
>> > This doesn't reflect support anymore?
>>
>> That is irrelevant. Why are you pushing so hard against this? Are you
>> trying to say that if users don't have any means of giving feedback to
>> the GNOME team, that is _good_?
>
> I'm not, I'm just checking as I'm getting confused with (in my view)
> the various topic changes.
>
> Initially you talk about support, then communication, then feedback.
>
> I've stated that we do not really have any means of support and wondered
> why the question is suddenly different.
>
> From another answer I gather you want some way to have people vote for
> improvement ideas which is I guess what you mean with 'feedback' or do
> you mean 'survey' as feedback?
I want to know if people feel they can give feedback, and I also want
to know if they feel they can ask questions somewhere.
Again, in the Git survey actual questions are:
24. Have you tried to get help regarding Git from other people?
25. If yes, did you get these problems resolved quickly and to your liking?
26. What channel(s) did you use to request help?
27. Which communication channel(s) do you use?
And the results are pretty interesting.
> Regarding voting: That was discussed on d-d-l a long time ago. In short:
> various concerns. Not against improvement, but in practice: We rather
> not improve things via internet voting, but more improve things after
> investigating what needs improvement based on actually seeing what does
> not work and fix it possibly in another way. For that you'd need to
> understand why something is asked for.
Yes. Nobody is saying you should mindlessly do something because
people vote for it (nobody does that). It's just another input.
> I'm not against changing things, I am not against a survey.
Good. I am not actually suggesting changes, but if users feel they
need communication channels, I would like that to be reflected in the
survey.
Cheers.
--
Felipe Contreras