Associate Member Judging Guidelines
The Associate Member Judging is the first step in a two-step process leading to the awarding of The Webby Awards winners.
Step 1: Associate Member Judging
Step 2: Nominee and Winner Selection
As an Associate Member, your task is to judge and rate submitted entries against the Academy criteria. Entries are judged by multiple Associate Members. Entries that fare the best will be passed on to the Executive Academy for consideration as a Webby Nominee and Winner.
If you have any questions, do not hesitate to contact Academy Director, Seth Callaway at email@example.com.
Schedule & Commitment
Each Associate Member will be assigned to a judging period of 2-3 weeks during which you’ll be asked to complete judging for your assigned entries. You may complete your judging at any time during your assigned session. You’ll receive an email from us with your schedule.
Please contact firstname.lastname@example.org you have any questions.
The judging site will guide you step-by-step through the process of rating an entry. It consists of the following steps:
1. Determine if the Entry Loads Properly
Click on the link; make sure it works. Be patient. Allow films and video to download. You may need to download or update software applications in order to view the entry. Test the username and password (always best to copy and paste) if needed.
If everything is working then answer “yes” to “Does the Entry Load?” If something is wrong, answer “no”. Please select one of the possible reasons the site did not load from the pull-down menu provided. In addition, feel free to send a brief note about what was wrong with the entry to email@example.com that we can look into the problem.
2. Check for a Conflict of Interest
Entries in a selected category will be randomly served to you for judging. Before you judge a given entry, we must confirm that you do not have a conflict of interest that might prevent you from objectively judging the work.
The Academy considers it a conflict of interest if you have:
- worked (in any capacity) on the development of the entry – contract or otherwise.
- worked for the company that developed the entry even if you didn’t work on the particular entry.
- worked for a direct competitor of the entry.
- any personal or professional bias for or against the entry.
For each entry, you will be asked to confirm no conflict of interest against the conflict criteria prior to judging the work. Answering “yes” disqualifies you from judging that entry. You will continue to be fed potential entries until you receive one with which you do not have a conflict.
3. Review the Entry
View the entry. Give it as much attention as required to accurately rate and judge it. Explore the content, visit the various layers, read the text, test out the tools. If it’s a video game, play it a few times. If it’s a videoblog, watch several episodes. Login and review the special content if there is a login and password associated with the entry. Please do not use provided login and password information for any other purpose besides Webby Judging.
Keep the criteria in mind.
Remember that some entries are frog princes whose value is not immediately obvious. Not all Webby-winning sites are visual 5s; frequently their value lies in the content they provide or the resources they allow access to.
4. Rate the Entry
Advertising & Media
Online Film & Video
Mobile Sites & Apps
Before you begin
“Warm up” and set your own benchmarks before you start judging. Check out the following Webby nominees and imagine how you might review them:
Flickr: great example of a site that displays excellence throughout all Webby criteria.
Rocketboom : Though the Visual Design is simple and typical of video blogging, Rocketboom utilizes equal strengths of talent, writing and production to boost itself to the forefront of the online video uh, boom.
The Huffington Post: Like Flickr, a stellar site across the board.
Clearly understand your 1, 3 and 5’s. This will help you remain consistent in the ratings you give.
- Stay present– Focus on the entry you are rating, the criteria, the benchmark values you have set, and your knowledge of the competitive quality of Websites, Advertising & Media or Online Film & Video in the category.
- Be consistent– Although we don’t want your benchmark to be the previous entry at any moment, be consistent in the value of a 1, 3 or 5.
- Save your top scores (5)– Remember as you go that when you give a top score (5), you should not run into an entry to which you want to give a higher rating. There can be a site that you think is equal, but not better. Of course, if an entry just can’t be beat, dish it out, and hear the crowd roar.
- Don’t penalize repeats– Some entries are entered in more than one category, and you may be asked to review the same site twice for different categories. Do not give the site a 1 or 2 simply because you’ve seen it before; instead, evaluate the site for the criteria of the category it was entered in.
- Don’t automatically give broken, or foreign language sites a 0 – Please mark them no load and use the notes field provided. Often times, we have translations that we can send you for sites submitted in foreign languages (we DO ask that sites be submitted in English).
5. Optional: Write a brief review
If you’d like to provide your thoughts and opinions for the entry, please feel free. Draft a brief sentence review that gives a snapshot of the entry and its high and lowlights. Be descriptive, and tell us something beyond what you’ve communicated in your ratings. Instead of writing, “The content is good,” explain why the content is good, as we’ve done in the example review below. You may want to focus on elements not already covered in the ratings or elements from the ratings that deserve further explanation. Be thoughtful and creative, but stay brief. Try to spend more time exploring an entry than you do writing your comments, should you choose to provide them.
Below are some sample write-ups, should you want some guidance:
“ThisSite.com is great, but not extraordinary–more worthy of a USA Today “Hot Pick” than a Webby Award. The graphics are a little dated, though still elegant and inviting. The interface is fairly intuitive, but the content is B-Grade at best. The last article dates to two months ago. Where this site really makes up ground is in its functionality – the site is fast, potential downloads come in a variety of formats and are well marked. If only the content was a bit more current.”
Great banners make you click on them almost immediately. With so much content on a page, your eyes should go right to the ad. And this banner is no exception. The mouse-over action was very inventive and I found myself actually spending time inputting information — into a banner ad! The ad wasn’t intrusive and was very creative. A simple concept that I’ve seen copied now many times. A great piece of interactive advertising with inventive creative and unique functionality!”
“Although I’m not Scorsese, I’ve worked on enough films — online and offline — to know a clunker when I see one. The plot was barely evident, the acting even more invisible and although the filmmakers were surely limited to a smaller budget (hence posting online) they could have gotten more for their dollars. The fact that the film isn’t a total bore is a credit to the producers and to their potential. But I’d definitely advise that the Webby be saved for a more deserving, more coherent film. ”
Have questions? Something changed? Something not working? Don’t hesitate to let us know.
|The Webby Awards &
The International Academy of Digital Arts and Sciences
22 West 21st Street, 7th Floor
New York, NY 10010
To reach other members of Webby Team, visit our executive team page.