RPG LAND Forum Index RPG LAND
Welcome to RPG Land's discussion boards. Most RPG heroes post here before they lose their memories and star in video games.
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Issues with site setup

 
Post new topic   Reply to topic    RPG LAND Forum Index -> The Site and the Forums
View previous topic :: View next topic  
Author Message
DunseDog



Joined: 21 Jan 2010
Posts: 7
Location: London

PostPosted: Fri Jan 22, 2010 5:43 am    Post subject: Issues with site setup Reply with quote

Please don't take this as a rant, but constructive criticism. If you don't agree with me you can explain why and I will respect your opinion, just as hopefully you will with mine (at least to some degree).


The "Out Of Ten" Setup for rating seems a little odd and no its not that your average is 5. That system is already in the majority of film reviews. However, many people get confused and expect your reviews to be the same as other game critics, averaging near 7. I think that if you changed to a 5 star rating system (using 0.5s in between and thus keeping 10 possible ratings) them this may make it more clear to outsiders. Furthermore, more people will respect something getting an 8 (Great) when that translates into 4 stars or how a 6 (Good) is 3 stars still makes the game quite enjoyable.

Some of the words used such as "Very Good" and "Great" can also be confusing. In my head "Very Good" is actually more positive a claim than "Great". In a similar style to my first point, you could also make some of the 6-10 rating words even more positive to reflect how essentially a 7 is still a good score, but evidently that's up to you lot and I wouldn't say its the biggest problem.

The category called "Replay Value" feels a little flawed to me. If a 25 hour game has the replayablility to demand a second play, its lifespan is no longer than a 50 hour game with no replayability. Hence, I believe this should be changed to "Lifespan" or something equivalent and some of the scores (if you want) to be changed to reflect this.

P.S If you can get more reviewers adding in second opinions of reviewers it might stop people complaining about writer inbalance as much, because it would either show a review's controversy with the staff or the agreement the staff have over it, etc etc.
Back to top
View user's profile Send private message
Slanderous!



Joined: 14 Jan 2010
Posts: 17
Location: New York

PostPosted: Fri Jan 22, 2010 10:24 am    Post subject: Reply with quote

Whenever I write things for this site, I more or less treat Replay Value as Lifespan. Even... if I didn't actually say it. I always figured that was what it was.
_________________
I do what I can.
Back to top
View user's profile Send private message AIM Address MSN Messenger
Strategos
a$$


Joined: 01 Feb 2006
Posts: 2100
Location: Alll ston, MA

PostPosted: Fri Jan 22, 2010 10:40 am    Post subject: Reply with quote

Your criticisms are too polite and well-worded to dismiss out of hand. Boooooooo.

Maybe I'm just a stubborn, elitist asshole, but I like the current X/10 system.
_________________
H-E-S-T-E-R is a long, convoluted joke.
Back to top
View user's profile Send private message Send e-mail Visit poster's website AIM Address
NR
The Great White Bear of Sunflower Street


Joined: 31 Aug 2005
Posts: 7614
Location: Japan

PostPosted: Fri Jan 22, 2010 12:12 pm    Post subject: Reply with quote

Thanks for your input. It's always appreiated.
Your opinions are very well thought-out, well expressed, and certainly respctable.


Quote:
I think that if you changed to a 5 star rating system (using 0.5s in between and thus keeping 10 possible ratings) them this may make it more clear to outsiders.

This has come up in the past. In some cases it seems like it might do that, but really, I've seen plenty of fallout and misunderstanding happen on /5 scales as well. If we hit Final Fantasy XIII with a 3.5/10, the internet freakouts would have been identical; I bet the farm on it.

Quote:
Furthermore, more people will respect something getting an 8 (Great) when that translates into 4 stars or how a 6 (Good) is 3 stars still makes the game quite enjoyable.

It's true that there's a problem with humans where they don't see the ratio, they often focus on the distance and I will agree that sometimes a 6/10 is not looked at the same way a 3/5 is. But when the chips are down, with the big releases like FInal Fantasy and such, a 3/5 is going to be largely received the same way.

Quote:
Some of the words used such as "Very Good" and "Great" can also be confusing. In my head "Very Good" is actually more positive a claim than "Great".

Words are subjective indeed. Numbers are even moreso (so we've learned over the years).
Not much to say here with regards to "Very Good" being somehow above "Great."
Perhaps our respective local dialects come into play, but I can't fathom "Very Good" being above "Great." Equal, sure maybe, but eh, I'm inclined to believe Good-Very Good-Great works fine.

But even that above is missing the real point. It's not about a ranking system, it's about telling the reader what we think of a game. If they read "Very Good," the goal is for the reader to walk away saying "Well, he or she finds the game overall 'Very Good.'"
If they read "Average," the goal is for the reader to walk away saying "Well, he or she finds the game overall 'Average.'"
If they read "Great," the goal is NOT for the reader to walk away saying "Well, he or she finds the game overall 'less than Excellent.'"
If they read "Very Good," the goal is NOT for the reader to walk away saying "Well, he or she finds the game overall 'less than Great.'"

And planning something based on that, I feel, is just setting oneself up for even more trouble. Really, if we switched to /5 and switched the meanings of Great and Very Good, it'd only be a matter of time till someone else emailed to posted on the forums with a better idea: switch to /10 and reverse the meanings of Very Good and Great (again).

Everything that doesn't conform to human stupidity is bound to cause problems once in a while, because people in general are terribly stupid. IGN, for example, gets little criticism and enjoys extremely high traffic to its reviews becuase low scores are extremely rare. They cater to the dumbness of humanity and succeed because of it. Anything else seems bound for failure. RPGFan, similarly, uses percentages, but anything below 80% is generally trash.


Quote:
In a similar style to my first point, you could also make some of the 6-10 rating words even more positive to reflect how essentially a 7 is still a good score

We already call a "7" "Very Good." Really, if someone fails to understand "Very Good," then not much else will help them. Our time is best spent writing reviews for the people who are smart enough to understand. If someone doesn't understand what a giant bold "Very Good" means, they need English lessons, which I do provide, at a price. If not, then it's not worth my time to try to cater to that audience.

In any case, we've moved on from numbers anyway. We going to stick to words and words only, out of ten. Review scores across the gaming world are a joke. Here, we just stuck them there because it felt like the right thing to do.

Quote:
P.S If you can get more reviewers adding in second opinions of reviewers it might stop people complaining about writer inbalance as much, because it would either show a review's controversy with the staff or the agreement the staff have over it, etc etc.

We have like 2 regular reviewers on staff (I'm thinking of Ivan and Janelle when I say this)....
If you'd like to go out and hire more and bring them here and pay them, by all means do so. But right now, I can't afford that Wink

It's really much, much, much, much easier said than done to say "Get more reviewers!" O...kay? By what, putting up an ad? No thanks.

If this were a different website, your advice would be spot-on, and you're correct in thinking it'd be NICE to have more reviews, more reviewERS, and more second opinions. But with a website written by a close group of friends...it doesn't work that way.

Thank you for expressing these concerns.
_________________
If you buy stuff from Play Asia, help support RPG Land. Thank you.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Sethimothy
*Accomplished* Bastard


Joined: 20 Dec 2005
Posts: 2174

PostPosted: Fri Jan 22, 2010 1:33 pm    Post subject: Reply with quote

THIS WEBSITE HAS VIDEO GAME REVIEWS?!?!
Back to top
View user's profile Send private message
Omegarage
Captain of the War Wagon


Joined: 14 Aug 2008
Posts: 1539
Location: all up in your grill

PostPosted: Sat Jan 23, 2010 2:32 am    Post subject: Reply with quote

This website doesn't have "reviews", Sethimothy. This site has "Game Judgements."

The problem that I see with current X/10 is that we're so conditioned by schools to believe that 7/10 or 70% = C or "average". 5/10 or 50% = F or ''failure''. In actuality, Rpgland is one of the few sites that really gets it right when it comes to rating using a 10-point system. 5 should = average, 7 should = above average, but not perfect.
_________________
"The teen passed out during the ceremony and the town decided to drown the cow."
Back to top
View user's profile Send private message
DunseDog



Joined: 21 Jan 2010
Posts: 7
Location: London

PostPosted: Sat Jan 23, 2010 8:22 am    Post subject: Reply with quote

"And planning something based on that, I feel, is just setting oneself up for even more trouble. Really, if we switched to /5 and switched the meanings of Great and Very Good, it'd only be a matter of time till someone else emailed to posted on the forums with a better idea: switch to /10 and reverse the meanings of Very Good and Great (again). "

Haha, I suppose that is actually true. Nevertheless, I present my ideas as, fundamentally, ideas, not as requests, therefore meaning that I only wrote this in case you hadn't thought through these ideas before, because they hadn't come to mind. Anyway, you are the site admin, so you will know the ins and outs better than me.

One problem I do see is that gamers wish to play the greatest of games. Now, this might be seem like "No shit Sherlock" but I will try to articulate what I mean. For films and music, the positive features of specific pieces of media are abstract to say the least. However, in gaming, the order, interactivity and structure of games make the general consenus on how good a game is a lot tighter than music and film. This means that it is far more likely that multiple reviews will have similar scores. Subsequently, a general consensus of a game being say an 8 is a lot more approachable and persuasive for a consumer, because they are more likely to believe the same things than in films and music (and especially art, but nvm). Hence, having such a large "looking" gap (just so you know I am trying to present the epistemology of the average joe, so don't react by arguing against me personal) between scores of fundamentally good games shocks their idea of the narrow consensus, making them discard the "slightly or quite above-average", instead they are always looking "masterpieces" and "near masterpieces". This comes into the fact that people react to FF13 getting a 7, that it isn't "the cream of the crop" and therefore only deserving of purchase with, let's say, extra disposible income.

Similarly, I wished that FF was gunna be a "masterpiece", because it has taken so long, but I remembered how I even enjoyed the ones that I thought have semi-flaws (*cough* FF12's story) because of hugely redeeming factors (*cough* FF12's combat (yes I am actually one of the people who love that game)).

Again this isn't a rant on your site, but I just wish to provide my own insight.


"Whenever I write things for this site, I more or less treat Replay Value as Lifespan. Even... if I didn't actually say it. I always figured that was what it was."

Yet don't you think that maybe the term dhould just be changed to help avoid confusion?


"Your criticisms are too polite and well-worded to dismiss out of hand. Boooooooo."

Damn, I'm sussed out. I thought your forum had enough arrogant trolls (I read the lovely story of Drakes_Fortune) and might want some politeness to offset the imbalance.
Back to top
View user's profile Send private message
Omegarage
Captain of the War Wagon


Joined: 14 Aug 2008
Posts: 1539
Location: all up in your grill

PostPosted: Sat Jan 23, 2010 8:50 am    Post subject: Reply with quote

You make points. Good points. I like you.

Drakes_Fortune wasn't as much a troll as he was a piece of garbage who couldn't man up when given the opportunity.
_________________
"The teen passed out during the ceremony and the town decided to drown the cow."
Back to top
View user's profile Send private message
NR
The Great White Bear of Sunflower Street


Joined: 31 Aug 2005
Posts: 7614
Location: Japan

PostPosted: Sat Jan 23, 2010 1:58 pm    Post subject: Reply with quote

Quote:
One problem I do see is that gamers wish to play the greatest of games. Now, this might be seem like "No sh-- Sherlock" but I will try to articulate what I mean. For films and music, the positive features of specific pieces of media are abstract to say the least. However, in gaming, the order, interactivity and structure of games make the general consenus on how good a game is a lot tighter than music and film. This means that it is far more likely that multiple reviews will have similar scores. Subsequently, a general consensus of a game being say an 8 is a lot more approachable and persuasive for a consumer, because they are more likely to believe the same things than in films and music (and especially art, but nvm). Hence, having such a large "looking" gap (just so you know I am trying to present the epistemology of the average joe, so don't react by arguing against me personal) between scores of fundamentally good games shocks their idea of the narrow consensus, making them discard the "slightly or quite above-average", instead they are always looking "masterpieces" and "near masterpieces". This comes into the fact that people react to FF13 getting a 7, that it isn't "the cream of the crop" and therefore only deserving of purchase with, let's say, extra disposible income.

Not quite sure what you mean here.

People want to play the greatest games and therefore we should...give higher numerical scores? Um...? No...?

If a game isn't "the greatest game," then we're not going to award it the greatest review. That's just being honest, which I take pride in being, even if it's not popular. I'm not like RPGFan or IGN, which will spend 75% of its text slamming a game's flaws and then give it an 8/10. It just doesn't work that way here.

Quote:
t it isn't "the cream of the crop" and therefore only deserving of purchase with, let's say, extra disposible income.

But see, it actually ISN'T the cream of the crop. It appears that way because that's exactly what the reviewers was saying. The key is reading the text as well.
Now, again like I said in my first post --we've gotten rid of numerical scores -- so the stuff about a 7/10 is outdated. We don't use numbers anymore. Words alone.

Anyway, again, if someone doesn't read the text and doesn't udnerstand a giant "Very Good" at the end of a review, then that person should not be reading, but should be attending language classes.

If they are going to rant and rave about it, it doesn't bother me because I know the person doing that is inferior to me. Sometimes it interests me, yes, because human psychology is fascinating; but it's nothing I get upset over. I got rid of numbers to clear up my inbox a bit, but eh, if people don't get it, they can read other websites that will baby them. Not me. The gods of RPG Land show minimal compassion.

That's why, when I murder someone, I usually don't feel guilty because I know I am the best person.
_________________
If you buy stuff from Play Asia, help support RPG Land. Thank you.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
DunseDog



Joined: 21 Jan 2010
Posts: 7
Location: London

PostPosted: Sat Jan 23, 2010 9:34 pm    Post subject: Reply with quote

NR wrote:
People want to play the greatest games and therefore we should...give higher numerical scores? Um...? No...?


Well again what I am trying to say is a representation of gamer psychology and their approach to reviews and why they aren't used to wide spreads, not some sort of reason for you to conform.

In my case, the way in which I found this site was a link to your FF13 review. I thought that the review was amazingly well written, when I assume your reviewer was playing the Japanese dub, yet I was puzzled by your ratings system. When I discovered why this was I thought it was very clever, especially as it gives you a unique selling point to your site. Nevertheless I thought how the internet behaves ("Funk this review, funk that console, funk..." etc) and might not understand. In the end I realised that your review actually gave FF13 an equivalent to a 8.2 and I realised that really isn't that bad.

This made me thing that maybe if your were to have your reviews included on metacritic (I dunno if you do) you could tell them how your scores are different and what your equivalent values would be. For example, Average as 7 and 0.6 intervals up and down (or thatever you think is equivalent). People may not actually be reading the actual important section of your reviews (the text) but it may allow for a little bit more cohesion with the rest of the internet.

Oh and my point about "Replay Value" being replaced by the word "Lifespan" still stand, unless you can think of any negatives of doing so.

Omegarage wrote:
Drakes_Fortune wasn't as much a troll as he was a piece of garbage who couldn't man up when given the opportunity.


Secretly, deep down, he must just know he is wrong. Maybe he thinks that if he rants and raves on gamfaqs that his dreams will become reality and a game will suddenly become better.
Back to top
View user's profile Send private message
VsPluckyDuck
Duckslit


Joined: 19 Jan 2006
Posts: 1268

PostPosted: Tue Jan 26, 2010 4:19 pm    Post subject: Reply with quote

DunseDog wrote:
NR wrote:
People want to play the greatest games and therefore we should...give higher numerical scores? Um...? No...?


Well again what I am trying to say is a representation of gamer psychology and their approach to reviews and why they aren't used to wide spreads, not some sort of reason for you to conform.

In my case, the way in which I found this site was a link to your FF13 review. I thought that the review was amazingly well written, when I assume your reviewer was playing the Japanese dub, yet I was puzzled by your ratings system. When I discovered why this was I thought it was very clever, especially as it gives you a unique selling point to your site. Nevertheless I thought how the internet behaves ("Funk this review, funk that console, funk..." etc) and might not understand. In the end I realised that your review actually gave FF13 an equivalent to a 8.2 and I realised that really isn't that bad.

This made me thing that maybe if your were to have your reviews included on metacritic (I dunno if you do) you could tell them how your scores are different and what your equivalent values would be. For example, Average as 7 and 0.6 intervals up and down (or thatever you think is equivalent). People may not actually be reading the actual important section of your reviews (the text) but it may allow for a little bit more cohesion with the rest of the internet.

Oh and my point about "Replay Value" being replaced by the word "Lifespan" still stand, unless you can think of any negatives of doing so.


A couple things:

-First, thanks for the review compliment! I tried to word hard on it and evaluate the game on its own merits.

-The problem with game review score systems doesn't originate from gamers themselves having certain expectations about scores, but more from other sites themselves saving face. Unlike the movie, book, or music industry, this is an industry that heavily relies on publisher support to keep reviews and sites rolling. Publishers don't like it when things get bad scores. So scores trended upwards over time on large sites that rely on or used to rely on publishers for their resources. There was artificial inflation that has had the long-term effect of narrowing the scope of the scale--to simplify a long story and complicated dynamic. Perhaps some part of the problem also comes from reviewers who lack perspective or context in some cases.

I wish that MetaCritic itself would take a stand and centralize their scale for games to make it equal to other scales on its site, but it would involve too much ambiguous score adjustment for them, and people don't like their scores getting tampered with anyways. The high-trending scores are just so ingrained in the outlets that it's passed on to the people who deal with those scores, like gamers or aggregates.

-I'm not going to bag on your estimate of what my review equates to numerically in your mind, because you were reading and digesting the text, but 8.2? It just feels so arbitrary to me, particularly with the decimal place. This is just a good example of why numbers are a problem in the first place. Our word scores correspond to numbers for aggregates only, and the "out of 10" is left in to "trick" people into being confused and stumble to our criteria page and gain some additional perspective. We know people struggle with our scoring system and that's why we semantically set it up the way we did.

As for an /5 or 5-star system, one could argue that it's the same as an out of ten system (using .5s) but approached differently by consumers. It's true to an extent, but a) even though it appears the same as /10 numerically, there are some negative semantic issues for a reviewer to deal with when categorizing, b) like Heath said, people will come to view them similarly anyways by converting to /10 or otherwise, and c) it muddles where the true average is--some consider 2.5 the average while others consider a 3 the average, causing the average to trend upwards and doing the exact thing we want to avoid.

I appreciate your well-thought-out points and responses. Some things you're saying are definitely applicable (if someone can come up with a synonym that can more clearly communicate the concept of "Great," I might be all for that), but for the most part, our scale is tailored the way it is with a lot of thought and reason behind it. You wouldn't believe the behind-the-scenes debates we've had! I'm really satisfied.

As for "Replay Value" vs. "Lifespan," I find that the latter doesn't communicate to me as clearly what's intended. It kind of has a connotation of the durability of the physical media, or how long it might be culturally relevant, or any other kinds of lifespan that a game might have. While Replay Value is confusing in that extra content not necessitating an extra playthrough is still considered under the umbrella of the score, I find Lifespan more ambiguous overall.

Is the description of each breakdown score on the criteria page? If not, it should be. For me a killer is Gameplay because it can encompass so much. There are a lot of hidden ups and downs that a simple score can't communicate. Again, the text is so, so key. This is getting a bit rambly, but you get the idea. At RPG Land, we really do care about trying to score fairly and really evaluate our own thought processes behind scores and review texts. (And then there are some of the truly, spectacularly awful old reviews in the archives that would seem to contradict this. Oh, Q Jets.)
Back to top
View user's profile Send private message
DunseDog



Joined: 21 Jan 2010
Posts: 7
Location: London

PostPosted: Sat Mar 27, 2010 11:12 pm    Post subject: Reply with quote

You make some good points. (sorry this is late, I only remembered to do back on cos of the FF13 release)

Firstly I just choose 8.2 mathematically from a standard average starting from 7, I didn't mean that you should write that as the score, just what it is equivalent to when comparing (such as on metacritic, which should standardise everything to the centre, I agree).

Similarly, if you look at the metacritic score, what's the average? 8.3, which essentially means that all haters of your review can just shut up right there and then. Obviously your opinion is subjective, but I think that credit is due when reviews aren't in the extremes, because they can be deceptive for the general reader.

Anyway, thinking about "lifespan" does come off as describing physically utility. Maybe something using the word "duration" (which connotes an exact time, not relevance or a period of animacy), like "appeal duration". I digress, that's for you lot to decide; there's no point having a discussion, here, over the exact words.

Maybe to demolish bands of scores could help what you were saying in the last paragraph. Instead, specific paragraphs (or sentences) could directly and obviously address those bands (maybe with a sub-heading) so that the issue of how fair reaching gameplay can be and how irrelevant, for instance, sound is sometimes can be raised and explained.

P.S *cough* Mass Effect 1 review *cough*
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    RPG LAND Forum Index -> The Site and the Forums All times are GMT + 9 Hours
Page 1 of 1
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group