Editorial: Smartphone Reviews Are Only the Beginning of a Larger Conversation

Note: This article is a revision of an opinion shared on the first site I wrote for. While the site is currently not up the original is cached and available. I hope you’ll give the original a read.

Well before a site called Fatal Hero launched in 2014, the staff struggled with the “right” way to handle game reviews. We didn’t believe any game can so easily be distilled into a single quote or digit like they often ended up in 2014.

Many years of hard work and development, focus groups, QA testing, and more go into nearly every title that graces our lives.

How can a fixed set of criteria be used to judge each one of them? To do so would be the equivalent of believing all games have the same set of parameters to be graded on, when the reality is that most games vary in play style, graphics, music, and overall creative vision. How could a review be so encompassing, yet only take into account a single viewpoint? How could we, the audience and players of those games, then take that point of view and treat it as gospel? What if the qualities of a game that I, as a reviewer, like are different than yours, as a reader? What if our expectations for a given title are different?

In the months that I have been contributing to XDA I’ve learned one valuable lesson: This argument extends beyond gaming and the scoring debate. We see it here every day – and as of late, our articles that praise, some that criticize and then others which clarify those criticisms of the Galaxy Note 7. The XDA forums have shown this from the very beginning as non-developers and enthusiasts often visit XDA, not just for development questions but answers on various situations for a device. As I reach an anniversary of starting to write online I find it’s time to extend this paradigm beyond its original scope. I hope that readers will consider this and truly look at it from the point of view that this debate deserves.

There Has To Be A Better Way

Here’s what it boils down to: Reviews – whether it’s a game, mobile device, application or even operating system – are mutable. That is to say, they are liable to change. The qualities that I look for in any of those categories are based on my experiences and what I enjoy. Likewise, what you (the reader) look for will change depending on your previous experiences and the specific object of review itself. We have become so fixated on trying to put a singular label or assignment on something that is meant to be a truly unique and individualized experience simply because it is mass produced. The multitude of combinations of apps and use cases my use of anything is likely not to exactly match another person’s entirely. Furthermore, the manufacturing process might be identical but that doesn’t always translate in every device functioning the exact same way — as seen with the battery issues and recent recall for the Note 7. So why do we expect a review to do this?

The Beginning Of A Conversation

I’ll say it again: Reviews are supposed to be the beginning of a conversation; not the beginning, middle and end. The launch or release is only a launching point from where we should be discussing the subject, talking about the various elements that we like and dislike. How can we assign a set process to review most things when each subject matter and, by proxy, our own criteria are unique?

The past few weeks have been a stark reminder that just because something looks great doesn’t mean it can or should be free of criticisms

The short answer is we cannot and should not. When I did reviews before XDA it was usually after an extensive use of the subject being reviewed. I’ve applied the same methodology here. Aquire the item on or shortly after launch day and share our experiences. Together. And continue to share that experience over time.  The item needs to be used, extensively, to really find out if it holds up over time or not. The past few weeks have been a stark reminder that just because something looks great doesn’t mean it can or should be free of criticisms. We, the group having the conversation, need to keep it in context though so that it doesn’t become unnecessary drama.

The best part: We do that largely for devices on XDA already! We have room to grow and expand that to other subject matter but it should always be to continue the conversation, regardless of the subject. Writers in general now have an unprecedented amount of interaction with their readership. At XDA are having the conversation with you as part of that community — in our forums, Reddit, Facebook, Twitter and other means. The most important part of all this is that none of this is possible without you.

Reviews Aren’t Fixed Anymore

It is our belief that we do ourselves, the subjects we discuss, or the community no justice by trying to write a conclusive opinion to something at the time we publish a review. In 99% of the cases that which we review is no longer the “fixed point” in time that cannot change. OS releases and contributions in the open source community, along with those challenging the everyday situations with a device, often yield results and positive change for the community. One of my first large scale interactions on XDA was the issue of the Galaxy Note and Galaxy S II bricking on wipes in CWM. It finally received an official kernel fix in no small part due to the discussion that originated in the community and then extended to communication by some of XDA’s finest with Samsung. We’ll continue to give you the good and what we believe is bad because it’s the same thing we would say if we were talking about that among ourselves in the forums. And also because we know that tech journalism and internet blogging does have an effect in the future of our devices.

At the time I wrote the original article for Fatal Hero this was not the norm. But the idea was in the right direction as other well-known gaming journalism sites started adopted similar policies. We won’t rush to a review deadline, sacrificing necessary time to use the review subject and possibly err in our reviews as a result. Instead we will respectfully walk a different direction than that and strive to keep our reviews the beginning of the conversation.

Each subject of review deserves its due and not just in an instant. The only way is to treat each review as unique as the review subject itself and, in turn, as unique as the community we represent. Our hope is to foster discussion and share experiences—just like when we talk about these things among coworkers and friends. The fact that others are now seeing and talking about these similarities help us remember what Robert Frost wrote long ago about taking paths less traveled. It still holds true to this day.

from xda-developers http://ift.tt/2csX8XR
via IFTTT