Aggregating the author… and The Google

May 8, 2008

At the end of the day, the reality every new author faces — even established mid-list and best-selling authors — is self-promotion. Getting the word out about their book. Branding their name. Driving eyeballs to a website that ultimately motivates readers to buy their product… it’s tough, hard work. Time consuming and rife with such brain damage as to mandate major pharmaceuticals, the bottom line is that, at least for most writers, shameless self-promotion is a creepily uncomfortable task.

But unfortunately, punching a hole through the cacophony of digital distraction that is today’s transmedia marketing world in effort to glean attention for one book, among the nearly 100,000 titles published each year in the U.S., is pretty much what every author seeking some calculable measure of success must do. And since author marketing strategies is something I want to address more substantively on this blog, I thought the below piece by SCWC*LA6’s Andrew Peterson (First to Kill) might make for a good kick-off:

Web Crawlers and Spiders and Bots, Oh My!

An Article on search engine mechanics

by, Andrew Peterson

I thought I’d write a brief article on the subject of “web crawlers” and “spiders” I hope you’ll find it useful. If you follow a few simple rules, you can maximize your internet exposure and move your ranking up in the search engine results and help keep yourself near, or at the top.

In a nutshell, companies like Google and Yahoo use search engine programs designed to scour the internet looking for keywords, web addresses, and traffic flows – both in and out of sites. These programs are called spiders or web crawlers. They are a specific type of bot, or software agent with a specialized purpose – to gather information from websites and index it in a HUGE database for recall.

It’s a super complex task due the dynamic nature of the ever changing World Wide Web. Think about it, how many micro changes are made every day to the tens of millions of websites out there? It’s staggering to comprehend.

That indexed database is then accessed when a user types a keyword into a search engine, like Google. For example, if someone types “Laura Benedict” (without the quotes) into Google’s search engine box, the program looks for all the instances where it finds the words “Laura Benedict.” The reason Laura’s web page appears, is because her name is associated with numerous instances where her web page address is also present. BTW Laura, you’re number one on the Google result page! You’re all over it! Kudos.

That’s why you’ll occasionally get a dead link in a search result. The search isn’t done live, it couldn’t be! It would literally take weeks to accomplish. The bots typically move through the web around once a week and they don’t look at more than 20% of the entire WWW. It’s just too huge to probe. But they do get a good cross section of the most current WWW at the time they look at it.

So here’s what you can do to increase your exposure: Whenever you blog in publicly viewed forums – like in MySpace, you should add your webpage to the end of comments you leave for other MySpace members. Here’s a simple example:

Hi Gilligan, loved the show, especially the episode where the ape threw the explosives at you! All the best, Andrew Peterson (Andy)


I didn’t have to say “please visit my website” besides, it rarely works anyway. When was the last time you looked at a website because the blogger asked you to? If the reader’s interested, he/she will look at it on their own, they don’t have to be asked. Make sense?

It’s not considered rude or improper or a BSP (Blatant Self Promotion) to include your website in a comment to a fellow MySpacer. It’s standard practice. What it does, is give the web crawlers another spot to find your name and your web address linked together. Remember every hit counts at moving you up in the search engine results. So whenever you’re in a public blog, forum, or you’re mentioned on someone else’s web page, always ask to have a link to your website included. It’s not an unreasonable request. Of course there are no absolutes, so you’ll have to use your best judgment when asking. To give you an actual example: When I first joined, I noticed the side bar of the ITW Debut Author page didn’t have my website linked, so I asked Kelli Stanley to add it, which she graciously did. Thanks Kelli!

That’s also why I included my web page at the end of the Southern California Writers Conference blog entry I made when Director, Michael Gregory did the write-up of my website. It looks perfectly natural to see a web address at the end of a blog entry. As long as you don’t get greedy and say something tacky about visiting your website, it’s perfectly okay. Some blog sites may have rules or restrictions on the website link issue, so all you have to do is ask if you can include it at the end of your blog entry. Most will probably say yes.

I’m in a tough situation. There’s another Andrew Peterson out there who happens to be somewhat famous. He’s a gospel singer and composer and he’s also written a book. So I’ve got a distinct disadvantage going. I’m hoping that will change after my book is launched and the spiders start finding reviews, blogs, and increased web traffic to and from my site. Until then, I just have to be patient. What’s really weird, is that Google has somehow “blacklisted” my site — it doesn’t even show up after 500 entries of search results. Yahoo has me on its first page, in seventh place, last I looked (which is actually pretty good considering FIRST TO KILL hasn’t even launched yet) I contribute some of that to all the networking I’ve done to date. My web designer is looking into the Google anomaly. For awhile, Google had me at number four! Then I just dropped off the map. Things like this happen, so search yourself often. (I know that sounds funny)

So, bottom line? Remember to add your web page (and in some cases, your book’s title) to all your internet traffic on blog sites, forums, and web pages within the public realm. You want those web crawlers and spiders finding you.

I hope this makes sense to everyone, and for those of you who already know all of this, sorry for the redundancy of knowledge. Now go forth and get bitten by those spiders!




  1. Michael, thanks for posting (WWW.EDWINDECKER.COM) this article.

    It was extremely (CHECK OUT MY MYSPACE) helpful.

    From now on (BUY MY BOOK DAMMIT!) I am going to employ this (WWW.EDWINDECKER.COM) strategy whenever posting on the (SUPPORT ME NOW PEOPLE!)internet


  2. Me too! And if the spirit of being fun and obnoxious:

    Buy my book, Jen-Zen and the One Shoe Diaries from Synergebooks.com. This author needs book sales otherwise she will start walking around with one shoe. Just a little humor in the spirit of my novel. You’d get it if you read it.

    And hey if you are cheap then enter my free book scavenger hunt at Gather.com. You could win my book for free. Visit this link for all the salicious details. http://www.gather.com/viewArticle.jsp?articleId=281474977335718

    Julie Ann Shapiro

  3. […] Aggregating the author… and The Google […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: