| Pages in topic: < [1 2 3 4 5 6 7 8 9 10 11] > | Kudoz - Leader list - A good moment to sort by usefulness rate too? Thread poster: Tomás Cano Binder, BA, CT
|
|---|
| | | Difference in practice | Aug 30, 2009 |
In practice the difference is however substantial, particularly for answerers with very low acceptance rates:
With [Kudoz points * acceptance rate^2], these get the same rank:
555 points and 75 % acceptance rate
1250 points and 50 % acceptance rate
5000 points and 25 % acceptance rate
With [Kudoz points * acceptance rate], these get the same rank:
667 points and 75 % acceptance rate
1000 points and 50 % acceptance rate
2000 points and 25 % acceptance rate
| | | | | Striking a balance between user-friendly and user-fair | Aug 30, 2009 |
I have been in the translation world since 1987, but have joined ProZ only yesterday (well, got scooped up by the pharma industry years ago and never had to worry about money and staying busy prior to this global recession ...). To the extent that I can judge the website thus far, it is a terrific tool for people in the language services business.
However, as I perused the ENG-GER and GER-ENG question lists, I was dismayed at the quality and confidence level displayed by a SMALL n... See more I have been in the translation world since 1987, but have joined ProZ only yesterday (well, got scooped up by the pharma industry years ago and never had to worry about money and staying busy prior to this global recession ...). To the extent that I can judge the website thus far, it is a terrific tool for people in the language services business.
However, as I perused the ENG-GER and GER-ENG question lists, I was dismayed at the quality and confidence level displayed by a SMALL number of posters. I believe our profession is too important and too laden with stigma to allow people to treat the search for an optimal rendition like crab shooting. But then again, if we try to solve every little issue with IT fixes, I am afraid the evaluation system will eventually become too cumbersome; it will end up being misused, abused and/or disused. Thus the need for a balanced approach.
Can we not use our existing ProZ system voices and call people out who make comments for the sake of making comments rather than to help advance fellow professionals. Instead of going "neutral" in our KudoZ responses, perhaps we should be more assertive and say "disagree". Isn't complaining about certain members afterwards disingenuous to some degree?
I believe that we won't be able to remedy each and every issue with a software tweak. At some point, integrity needs to be communicated directly from us, the professionals. I will try to keep the obvious "crab shooters" at bay. ▲ Collapse
| | | | Yaotl Altan 
Mexico
Local time: 00:10
Member (2006)
English to Spanish
+ ...
Ok, Bjørnar everybody seems to support this magnificent idea, but when will be implemented this new sort system?
Have a nice day.
| | |
|
|
|
Turz
Germany
Local time: 08:10
German to Italian
+ ...
| Simple or complex formula? The computer will do it | Aug 31, 2009 |
I think it's more fair to use the squared value, as in Bjørnar's proposal. The computer will do the calculation anyway, we just read the result.
My proposal would be to keep the option of ranking based on KudoZ points only, but making Bjørnar's proposal as the new default ranking.
Please remember that KudoZ points will not disappear, so user's activity will be rewarded.
For instance, I would be anyway far behind some notorious point-grabbers, even with the new ranking system, because of my low activity. But the new ranking would be IMO more fair.
| | | | | Good suggestion! | Aug 31, 2009 |
I agree that Bjørnars suggestion for a new way or alternative way of sorting the Kudoz leader list is a good idea. I hope the team at Proz will look further into how this could be done and also take steps to actually implement such change.
| | | |
Using acceptance rate in ranking would discourage people to answer questions which already have several answers, very often it would be a pity as frequently already given answers and ongoing discussion stimulate good ideas.
Btw. I think that this problem could be partly rectified by taking into account agrees and disagrees - after all they also indicate usefulness, what do you think?
Best Regards
Stanislaw
| | | | | Why would it be discouraging? | Sep 1, 2009 |
Stanislaw Czech wrote:
Using acceptance rate in ranking would discourage people to answer questions which already have several answers, very often it would be a pity as frequently already given answers and ongoing discussion stimulate good ideas.
Why do you think it would discourage people?
If there are several answers already, then I would look at them, and if I see the right answer, I would use the Agree option. If I don't see the right answer, and I have my own suggestion that is different from all the others, and I have supporting evidence, why would I refrain from posting it?
On the other hand, using the acceptance rate in ranking may discourage people to post an answer that is only a slight variation of an already existing answer (some call it "piggybacking"), which is probably a good thing.
It may discourage people from posting "answers" that are not really answers, and should be posted as References, which is again, a good thing.
Ongoing discussion can be conducted in the Discussion area, and contributions made there are not related to points and acceptance rate.
Katalin
| | |
|
|
|
| It's matter of probability | Sep 1, 2009 |
Very often there is no one best answer and from asker's point of view it is best to receive as many answers as possible (of course reasonable ones) however if there are already 4 answers it means that the chance that the fifth answer will be selected is just 20% - probably I wouldn't take such a risk.
As an asker I had several situations when there were at least two good answers and I would prefer to split points between two answerers rather than choose one answer however it's not p... See more Very often there is no one best answer and from asker's point of view it is best to receive as many answers as possible (of course reasonable ones) however if there are already 4 answers it means that the chance that the fifth answer will be selected is just 20% - probably I wouldn't take such a risk.
As an asker I had several situations when there were at least two good answers and I would prefer to split points between two answerers rather than choose one answer however it's not possible.
I believe that taking agree/disagree ratio into account would alleviate the problem as people who provide useful answers are likely to have many more agrees than disagrees.
Best Regards
Stanislaw ▲ Collapse
| | | | | You dont have to take the risk | Sep 1, 2009 |
Stanislaw Czech wrote:
Very often there is no one best answer and from asker's point of view it is best to receive as many answers as possible (of course reasonable ones) however if there are already 4 answers it means that the chance that the fifth answer will be selected is just 20% - probably I wouldn't take such a risk.
The 20% probability ratio assumes that your answer is no better ("better" in the sense that the asker perceive it as more helpful) than the other 4 answers. If that is the case, maybe you don't have to launch it as an independent answer? If it builds upon the effort of one of the earlier answers, you can mention your own version in an agree comment or in the discussion area. This happens all the time in my language pairs.
A usefulness rate will not restrain valuable contributions, just change the form they are delivered in.
[Edited at 2009-09-01 06:41 GMT]
| | | | Turz
Germany
Local time: 08:10
German to Italian
+ ...
| I take the "risk" when it pays off | Sep 1, 2009 |
ever if there are already 4 answers it means that the chance that the fifth answer will be selected is just 20% - probably I wouldn't take such a risk.
[/quote]
I took such a risk sometimes because I thought that my answer was the best, although the fourth or fifth. It didn't improve my acceptance rate, but it didn't worsen it either. Anyway, it improved my KudoZ points, and remember that absolute KudoZ points are still an important part of the equation.
I believe that taking agree/disagree ratio into account would alleviate the problem as people who provide useful answers are likely to have many more agrees than disagrees.
I have to disagree
It would make the problem even worse, because if there are already 3 or 4 answers there might already be some agrees, which are based on the (wrong) assumption that there's no better answer.
| | | | Attila Piróth
France
Local time: 08:10
Member
English to Hungarian
+ ...
| Against complicated formulas | Sep 1, 2009 |
I am in favor of Tomas's suggestion, i.e., to have a second list, in which users who have submitted a certain number of answers are ranked by the acceptance rate alone.
While the idea to balance high acceptance ratio and high number of answered question by means of a formula that takes both into account is interesting, I do not see it practical for a number of reasons.
First, the relative weight of contribution and acceptance rate may very well depend on the language ... See more I am in favor of Tomas's suggestion, i.e., to have a second list, in which users who have submitted a certain number of answers are ranked by the acceptance rate alone.
While the idea to balance high acceptance ratio and high number of answered question by means of a formula that takes both into account is interesting, I do not see it practical for a number of reasons.
First, the relative weight of contribution and acceptance rate may very well depend on the language pair. The "average acceptance rate" is quite different in large language pairs and in small ones (simply because questions usually receive more answers in the former than the latter) -- so a "one size fits all" approach would not work well.
Second, any odd formula can be calculated if the acceptance rate / reliability ratio is given. (Of course, the converse is true: given a formula plus the number of KudoZ points, the reliability ratio can be calculated as well, but one way is definitely simpler than the other.) And this difference in simplicity surely matters. Upon seeing a list where users are tied with 555 points and 75 % acceptance rate
1250 points and 50 % acceptance rate
5000 points and 25 % acceptance rate many people may easily get perplexed as to why there is a tie? And why this listing is relevant?
Since the pure acceptance ratio list is easy to understand and quite obviously relevant, as expressed in this thread by many contributors, this should be used.
Having the AR displayed optionally would help me, as a freelancer, show my expertise more clearly. Having an option to list translators according to it would help me, as an outsourcer, to select translators that showed expertise in the terminology of a particular field, so implementing such a feature would be perfectly in line with ProZ.com's mission statement of "providing tools and opportunities" for translators.
A recent implementation addressed the listing of agencies according to their average LWA score: in the right-hand column of the home page, you can see a box with the title "Blueboard ranking". The ranking criterion changes, "More than 100 entries, all time", "More than 30 entries, last 12 months", "More than 40 entries, all time", "More than 15 entries, last 12 months" etc. With this customized rotating display the ranking is according to a certain "reliability ratio", while contributions are also encouraged.
In the KudoZ leader board, the language pair and the specialty field can be selected. By adding options to list answerers according to the AR and to specify the minimum number of answers (or points), a very useful tool would be obtained.
The question whether this would affect traffic is a legitimate one. My take is that it would not significantly: piggybacking may decrease, but this could be offset by the participation of subject-matter specialists. It can also affect the quality of the answers, as pointed out by several others.
Admittedly, these are assumptions -- only implementing this feature would give a real answer to the question. Still, the assumptions derived from our KudoZ experience are interesting -- and probably worth a separate thread. Please consider contributing to this one: How would KudoZ participation change if listing according to the "acceptance rate" were introduced?
Attila ▲ Collapse
| | |
|
|
|
| Trying to keep silent | Sep 1, 2009 |
I am following the discussion in silence, but you are forcing me to come out of my cave!
Although I thank you all for contributing to this very interesting topic, let's face it: discussing all this will be like discussing the gender of angels; God will not change the gender of angels (if they have it), no matter how long we discuss it.
The same way: Proz.com does not care about this discussion. It's sad bu... See more I am following the discussion in silence, but you are forcing me to come out of my cave!
Although I thank you all for contributing to this very interesting topic, let's face it: discussing all this will be like discussing the gender of angels; God will not change the gender of angels (if they have it), no matter how long we discuss it.
The same way: Proz.com does not care about this discussion. It's sad but it's just the hard facts. They will never implement any measure like this, and they have said it a hundred times over the last years.
Do you really want to invest your time in a discussion that will lead nowhere? ▲ Collapse
| | | |
Tomás Cano Binder, CT wrote:
The same way: Proz.com does not care about this discussion. It's sad but it's just the hard facts. They will never implement any measure like this, and they have said it a hundred times over the last years.
Do you really want to invest your time in a discussion that will lead nowhere?
I suppose this is meant ironically? If not, why on earth did you start this topic?
| | | | | All ranks are language dependent | Sep 1, 2009 |
Attila Piróth wrote:
First, the relative weight of contribution and acceptance rate may very well depend on the language pair. The "average acceptance rate" is quite different in large language pairs and in small ones (simply because questions usually receive more answers in the former than the latter) -- so a "one size fits all" approach would not work well.
Attila
The same issue applies to total KudoZ points - it is normally easier to collect may points in large language pairs simply because more question are asked. There does not exist any rank that makes sense across language pairs.
| | | | | Pages in topic: < [1 2 3 4 5 6 7 8 9 10 11] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Kudoz - Leader list - A good moment to sort by usefulness rate too? | Trados Studio 2022 Freelance | The leading translation software used by over 270,000 translators.
Designed with your feedback in mind, Trados Studio 2022 delivers an unrivalled, powerful desktop
and cloud solution, empowering you to work in the most efficient and cost-effective way.
More info » |
| | Anycount & Translation Office 3000 | Translation Office 3000
Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
