When is a Review Not a Review?

Written by TheBBPS

Friday, 04 May 2007 14:37

At the end of March, I flew out to California to attend a pair of preview events, and I got a chance to check out a few builds of Spider-Man 3 at Activision HQ in Santa Monica. Along with the other writers at the event, I spent about 50-60 minutes between the Xbox 360 and PlayStation 3 versions, which are largely the same.

When I got back to the midwest, I culled together my notes and wrote a preview. Considering my limited time with the game (and the fact that it was still several builds away from being complete), there was no way I could offer a definitive opinion on the game.

A writer from PGNx was also at the event, and when he got back to wherever it is that he lives, he also wrote a preview. And then today, the website posted his review of the game. When I read it, the first thing that popped into my mind was: Wow, I could have written that based on the 50 minutes I spent with the game in March.

My hunch? I think that's what he did.

If you compare his two articles, you'll see a lot of recurring details and quotes (such as, "Spider-Manâ€™s character model is also more detailed, shinier, and nimbler than he has ever been. The same applies to Sandman and Scorpion..."). More significantly, there's nothing in his review that couldn't have been gleamed from the preview event and/or the previews posted elsewhere by other writers. The general lack of detail in his review signifies to me that he merely reworked his preview and added a handful of vague opinions to come up with what the site is boasting as the "first review online."

I could be completely wrong on this -- perhaps they received review code for the game ahead of time (as sites like IGN and GamePro.com have) and he spent some time with it on a debug before putting together this incredibly vague review. I can only go on my hunch and what I know from being at the event, but I wouldn't bother posting this if I didn't think it was most likely correct. I'm not down with shit-talking or sensationalism.

The writer seemed like a really nice guy, but the more I interact with some of the writers at these independent gaming websites, the more I sense a general lack of knowledge pertaining to ethics and correctness in reporting. It may not be intentional, but the end result is the same, and readers are stuck with incorrect information or uninformed opinions.

Check out the links and drop some thoughts in the comments.

Comments (17) Add Comment

...
written by B, May 05, 2007

"These interactive cinematics provided some of the demoâ€™s biggest â€œwowâ€ moments such as struggling in the air against the Green Goblin (while playing as Peter) and jumping through lasers as Spider-Man"

Looks like someone forgot to take out the word "demo" after a bad copy and paste job.

So you're completely right, he reviewed the demo and passed it off as a review of the actual game. Lame.

...
written by Andrew Hayward, May 05, 2007

Good catch! I didn't see that it said demo in there, but that was one of the key statements that made me think he "reviewed" the demo, as those two sequences were the exact ones shown to us at the event.

Pretty sad.

...
written by Andrew Hayward, May 06, 2007

Man, this is just embarrassing. They posted a review of the DS version (which we played at the event) that ends with the following quote:

"Weâ€™re very excited to review the full version when it is released in early May."

Wow.

...
written by CelShader, May 06, 2007

"I sense a general lack of knowledge pertaining to ethics and correctness in reporting. It may not be intentional, but the end result is the same, and readers are stuck with incorrect information or uninformed opinions."

Sorry, I agree that this guy is most likely not writing a review for the final game, like you have surmised. But couldn't you apply that same phrase I quoted above to THIS post? And most blog posts based on rumor...

...
written by Andrew Hayward, May 06, 2007

I'd have to disagree with you on that one, though I certainly understand where you're coming from.

I suspected wrongdoing at a publication that was making sensational claims, and though I certainly did not have undeniable proof, I presented my points on a public forum that has been granted to me. As noted in my post, I had a similar experience to that of the review writer (having attended that same preview event), but our paths diverged at that point. I wrote a preview, and will only be writing a review should I be sent a finished review copy, but he apparently wrote both articles based on the limited demo shown at the event.

In journalism, there is often a struggle between being first and being right. Balancing the two is a tricky situation, because being first may get you more readers (initially), but it increases the chances of you being wrong, which will only hurt your credibility in the long run. On the other hand, sometimes you have to take a chance based on solid information (and perhaps your gut feeling), even if you, say, couldn't prove it in a court of law. I'd hate to think where we'd be today if Woodward and Bernstein waited until Nixon said, "Yeah, I was on that" before attempting to bust Watergate wide open.

As I said to a buddy (who was also at the Spidey event) after posting this story, the worst case scenario here was that I brought attention to a poor review. I wouldn't be losing sleep if that were the case (though I might feel a bit silly). But in light of what readers like you have said, there is no doubt in my mind that the "review" is merely a reworked preview.

...
written by B, May 07, 2007

I'm glad you pointed it out. Most people wouldn't have the balls to call out another "journalist's" abuse of getting to play a preview then later on passing it off as a review. That being said, comparing the PGNx "review" with the other reviews out there, the PGNx one doesn't touch on any problems that the other reviewers have said. Like the wild camera. I played my little cousin's and the camera is so erratic that I couldn't control crawling up walls at times.

...
written by BBPS calls out rewritten previ, May 08, 2007

[...] Bits Bytes Pixels and Sprites is calling out PGNx media and writer Jose Liz for writing reviews based on previews. In the piece, Andrew Hayward of BBPS, says that the Spider-Man 3 “review” is based on a demo he saw with this writer while on an Activision junket. He believes the writer made up the review based on the preview information as evidence by lines like “These interactive cinematics provided some of the demo’s (emphasis ours) biggest ‘wow’ moments such as struggling in the air against the Green Goblin (while playing as Peter) and jumping through lasers as Spider-Man.” Oops, helps to double-check that stuff before posting. Coincidentally, the DS review by the same writer ends with, “We’re very excited to review the full version when it is released in early May.” Double d’oh! [...]

...
written by BBPS calls out rewritten previ, May 08, 2007

...
written by Xav de Matos, May 08, 2007

Tracked back from JoyStiq... seems like you guys are getting your due credit for truthful and information news items.

...
written by Jose Liz, May 08, 2007

Hi Andrew,

Thanks for taking the time to read our Spider-Man 3 coverage. You are completely wrong when you assume that I simply assigned the score to the press demo we were given back at Santa Monica. I received final, retail copies of all six versions of SP3 Wednesday, April 25th. In the ten days that preceded the review, I completed the story mode and more in three versions of the game (Xbox 360, Wii, and NDS) and the story mode for the remaining platforms. There was a definite time crunch in getting all six reviews up in 10 days and specifically that Friday, which is why I had to reuse some lines from the preview. I didnâ€™t foresee this to be a problem, although in retrospect, it does come across a bit carelessly. Thanks again for bringing this upâ€”Iâ€™ll make sure to copy-edit more thoroughly in the future.

Regards,

Jose Liz
PGNx Media

...
written by Jose Liz, May 08, 2007

As a side note, I think itâ€™s ironic that you blast someone else for using â€œincorrect information or uninformed opinionsâ€ when you did the same thing above. You could have easily called Activision and mentioned â€œHey, I think someone reviewed your game without playing the final version. You would have been corrected. I would be indebted if you could edit the original post to criticize our â€œvagueâ€ and â€œpoorâ€ review and not claim that we did not have final copies of the game as you imply in the original post and explicitly mention in the following comment: â€œhe apparently wrote both articles based on the limited demo shown at the event.â€

I look forward to reading your review when you receive final copies.

...
written by Chris, May 10, 2007

Jose,

I read your review, and it's fairly useless. Whether you did or did not finish the game before completing your review, while obviously important Mr. Hayward, is somewhat beside the point to me as a reader. Playing the game for five minutes at a GameStop kiosk yields enough of an experience to belie nearly every claim you made in your review. The frame-rate and fill-in problems with this game are so completely crippling to its gameplay and graphical integrity, that either you clearly did not give the game a honest and fair review, or that your knowledge of gaming is so poor that you shouldnâ€™t be reviewing even Barbie titlesâ€¦

In addition, thereâ€™s nothing ironic about anything Mr. Hayward claimed above, as he made both implicit and explicit using words like â€œhunchâ€ and phrases like â€œI could be completely wrong on this.â€ You're obvious inability to read and comprehend the simple point he was making bolsters my claim that you probably ought not be publishing anything at all.

Best of luck to you,
Chris

...
written by Justin, May 10, 2007

Josie,

PWNT.

Best regards.

...
written by Rob, May 10, 2007

burn

...
written by Tildy, May 11, 2007

As Chris noted above, the original poster made it very clear that he was speculating and did not have proof. Jose should firstly better comprehend what is written, and then refrain from attacking a reasonable poster in a childish, sarcastic manner. This is the internet, let's act accordingly ;q