GuruBootcamp: SEO (Video)

Adam Audette | Apr 4, 2011 | 0 Comments

While search engine optimization (SEO) as a channel is maturing and growing beyond technical competence only, there will always be a great deal of technical attention necessary in the work.

Some of the biggest problems negatively impacting SEO campaigns can be traced back to technical issues, since by nature they tend to hit areas around crawling, accessibility, redirects, and indexation. These are core, foundational areas that every hard working organic search marketer needs to be familiar with.

Having seen the results of technical work, I can attest first-hand that some of the best SEOs in the world are highly technical people. They have that unique combination of technical acumen and marketing savvy, a rare thing to find.

Most professionals realize that attention to the technical side is important in SEO. But where do you start?

What types of areas should you be investigating, and why? What factors really matter? What tools should you use?

I recently presented on this topic at SES New York alongside Brian Ussery, Brett Tabke, and Jaimie Sirovich, so this is especially timely for me.

What Technical SEO Factors Matter?

With so many potential variables at stake, where does one begin in technical SEO? And additionally, what factors matter the most? In our experience, technical work can be focused primarily on the crawling and indexation aspects of SEO.

Conceptually, SEO work can really be boiled down to three primary “chunks”: crawling, indexation, and ranking.

While there can be technical issues at all three levels, our primary concern is with crawling first, and indexation secondly. However, indexation is really secondary to crawling: without a site that is accessible and easily crawled, nothing will be available for indexation. For this reason, we focus first on crawling and accessibility.

The crawl experience is a first-order effect, and everything in SEO comes after that experience. (In reality, the crawl index rank concept is quite circular, because each component influences the other, but for the purposes of this discussion we’ll treat them as separate and distinct.)

I’ve given AudetteMedia’s list of SEO factors before in my article on SEO audits. It’s helpful to provide that here again, in order to demonstrate the sheer magnitude of potential issues. There is a lot of stuff to think about!

The key, however, is that while lists of potential factors like the above can be helpful guidelines, there is no recipe for technical SEO analysis. It’s always different, based on the site at hand, the problems being worked on, and the technology in question.

Therefore, instead of thinking of every possible component that could be a factor, think of these high-level factors as the most pressing and highest impact with regards to technical SEO:

• URL structure, especially characters and parameters: The URLs represent all the value for SEO, and they are often the culprits for de-valuing a site, too. Tracking parameters, session IDs, and other gremlins can cause duplicate content and crawling problems. Look carefully here.

• Crawl efficiency and accessibility: How efficient the site can be crawled, including ensuring URLs are structured cleanly with strong canonical signals, is key. Also, how accessible a site is, without overuse of JavaScript menus and AJAX, Flash, and other barriers, is important.

• Speed: How fast is your site? If you’re a big one, it’s especially important to be fast. Fast is good.

• Navigations and crawl paths: While similar to crawl efficiency and accessibility, crawl paths are really the pathways given to a search engine. Are important URLs prominently placed on the site? Are navigations smart and not simply boilerplate and the same on every single page type?

• Content structure and code order: How clean is your code, and does unique content get sorted highly, above all the other stuff?

• Content delivery and caching strategy: Having a smart solution for content delivery is essential for large, high traffic websites.

• Meta data: Ironically, meta data is now an important part of technical SEO, with the rise and power of rel=canonical.

• URL exclusions with robots.txt and meta robots: It’s important to consider what robots exclusion is taking place on a site with meta robots and robots.txt rules.

While there are many other areas to investigate, the above eight points represent key areas that are fundamental to a site’s technical SEO.

Tools for Technical Analysis

The cornerstone of solid technical SEO work is experience with solid toolsets. This is where you’ll find a multitude of resources and options, because every practitioner will have her favorite tools for the job. Here are mine, and I’d love to hear from others in the comments with other recommendations.

Remember that technical SEO work is really about correctly diagnosing (sometimes complicated) web requests, masking as another user agent such as Googlebot or Safari, or as a user in another country such as Italy or South America. For that reason, I always prefer to use the hardcore tool: the command line shell.

Most SEOs wince at the thought of firing up the command line, but remember that when working on technical projects, it’s all about time and efficiency. Because of the nature of the work, hours (even days) can be spent looking at specific things and working over data, only to find that the real issue at hand wasn’t actually related. When you have the ability to quickly open the shell and fire a couple of commands before looking deeper, it can save a lot of time.

Two of the most often used commands for fetching a URLs response headers are wget and curl. Both are quite powerful and very fast. Here’s an example of issuing the command on a site:

% wget -R –spider www.mydomain.com

This command tells wget to fetch the response headers and mask as a spider (thereby saving the time and space to fetch the entire file and save it locally). Without the spider argument, you’ll have a lot of locally saved files for pages for which you’re just concerned with checking the response headers.

wget is a powerful tool for checking things like multiple, chained redirects.

When you need to check a single URLs status code response and don’t care about following redirects, curl is a fabulous and elegant solution. The following syntax will return status code for a URL (please note the argument given is an upper case ‘ i ‘ not a lower case ‘ l ‘):

% curl -I www.mydomain.com

Curl, too, can mask as another user agent such as Googlebot:

% curl -I -A googlebot www.mydomain.com

This is a great way to find conditional redirects, such as the one demonstrated here (user agents visiting this site without the ability to accept cookies get 302 redirected):

When problems are sniffed out and investigated with the command line, it is then most efficient to use a graphical application for deeper analysis. My preferred choice for deeper technical work is Charles, a web proxy tool that serves as “man in the middle” between a web server and your browser. Charles captures everything happening between the server and the client, and records it in organized fashion.

When you need to examine chained redirects to drill into the responses, this is a great tool to use. It’s also quite useful when looking for things like conditional redirects (when a different response is given based on the user agent).

Such an example is demonstrated below. While browsing this site as Googlebot and then again as Firefox (using the User Agent Switcher in Firefox), different responses are recorded. Below is the response for Googlebot:

This is the type of technical SEO investigation that can have a big impact on a site. In the above example, the entire subdomain is unable to be crawled and indexed if cookies aren’t enabled (search engine bots don’t accept cookies).

Because of this implementation, the subdomain is basically invisible in organic search, although there are quality pages on the site. Ouch!

SEO Crawlers

An article could easily be dedicated to just the topic of SEO crawler tools. I’ll be brief and just mention a few here that can be useful tools.

• Xenu is a great little crawler that isn’t built specifically for SEO. However, it can do some great things and scales up to about 10,000 URLs or less. Anything larger will overwhelm it, so it’s best used for smaller, focused crawlers.

• IIS SEO Toolset is a fabulous tool by the good folks at Microsoft. There is a ton this tool can do. I recommend reading this excellent tutorial.

• 80legs.com is a powerful, web-based solution and can be configured to do powerful crawls.

• SEO Scalpel, while still in beta, is AudetteMedia’s own tool. It’s designed to be a powerful crawler that can be configured with regular expression to literally pull any field off a web page, including things like “content robustness” (crawling for thin content pages), analytics tags for any package, any meta data or anything in source code.

• Screaming Frog is a tool I’ve heard good things about. I have yet to test it myself, but am intrigued by the tool and hope to give it a run soon.

Log File Analysis

Log file analysis is a critical component of technical SEO. However, deep analysis is often difficult because of the size of the data (log files for big sites are ridiculously huge) and the problem of parsing out just what you need.

That’s why we developed a command line tool for this job called logfilt. It’s a SEO log analysis tool and is free to download.

Logfilt is written in sed and nawk (there is an awk version, too) and is tremendously fast. It is designed to be efficient and minimalist. You can parse through large data sets easily — the only thing holding you back will be your machine’s memory (for big data sets, run this tool on a big box).

One of the benefits of doing log analysis from the command line, is that you can put together very complex searches using regular expression. Graphical log diagnostic tools such as Splunk and Sawmill are also highly recommended, and can do the same types of searches (with custom configuration). However, if you have technical chops and aren’t afraid of the command line, doing the work there can save time.

Please see my site for a complete description on using logfilt, but briefly, here is an example of the syntax you would use to find all 302s reported by Bingbot in a specified period of time, and save them to a file:

% logfilt -H bingbot -R 302 big_log_file 302.txt

To return all URLs crawled by a user agent (Googlebot in this case) over a specified time period, issue:

% logfilt -H googlebot log-Apr-1 log-Apr-2 log-Apr-3 | less

(You’ll note the above example is piped to the command less, which is handy for pagination in the shell. You could just as easily save the output to a file.)

One of the cool features of the second example above is the output. The crawl is given chronologically, showing the exact crawl path of the user agent by individual URL. This can be a powerful set of data to examine.

Below is an example of the same command syntax, using Yahoo Slurp (now deprecated in the U.S., but still actively crawling internationally).

Bonus Tip: Crawl Your Top Pages

There is much more to write, but in the interest of time and space we’ll save that for another installment. Stay tuned for a future article where I’ll discuss log file analysis for SEO purposes.

Before departing, however, I wanted to leave you with one easy win.

1. Export your Top Pages in Google Webmaster Tools, under the “Links to your site” report. These are the most linked to URLs on your domain according to Google.

2. Pop those 1,000 URLs into your crawler of choice and select 1 level deep as your preference (so as to crawl only those URLs and no others).

3. Sort the report by status code and make sure they all 200. If you find any 302s or 404s, act accordingly.

4. Toast yourself for an easy win and celebrate good times!

Contact Adam Audette
Articles by Adam Audette

View print-friendly version
Send article to a colleague

Contact SEW Editors

• Share
• 
• 
• 

Newsletter signup

Adam Audette is CEO and president of the organic search marketing boutique, AudetteMedia, Inc., who’s clients include Zappos, University of Phoenix, Charming Shoppes, Kroger, Rockler, Primedia, and more. AudetteMedia specializes in advanced SEO and works discretely with some of the largest brands in the world. A frequent speaker at search conferences nationwide, Adam has been active in the search marketing industry since 1996. You can read his blog at http://www.audettemedia.com/blog.

Article Archives by Adam Audette

Maximizing Your CTR for SEO in Organic Results - Mar 7, 2011

Mea Culpa: How I Failed At Link Building - Feb 7, 2011

Quit Obsessing On Anchor Text Already - Jan 10, 2011

The Art and Science of SEO Site Audits [Best of SEW 2010 #10] - Dec 20, 2010

More article archives

Archive

Would-be robbers try and smash down Coulsdon bank doors

1:48pm Monday 4th April 2011

• Print
• Email
• Share
• Comments(0)

These pictures show the damage caused by would-be bank robbers who tried to smash their way into Santander on Brighton Road in Coulsdon last week.

Two suspects tried to smash in the door just after security men entered the bank to replenish the cash machine last Wednesday at about 7.45pm.

Armed with a sledge hammer, they began to hit the glass to try and smash into the bank.

They could not get through the glass and so ran off empty handed.

The men have been described as black and wearing dark clothing.

BT Tower-sized skyscraper planned for Croydon town centre

5:13pm Monday 4th April 2011

• Print
• Email
• Share
• Comments(0)

A gargantuan skyscraper rivalling the BT Tower in height could dwarf homes near East Croydon station, a councillor has warned.

The 53-storey Menta tower, which would help create almost 500 high-rise homes in the centre of town, will be taller than the NLA Tower and Nestle building combined if planning councillors give it
the green light in July.

Developer Menta describes the building as an elegant landmark housing a “vertical community”, and hopes to build a neighbouring 165-bed 4-star hotel and 6,600 sq m of high class office
accommodation at the site in Cherry Orchard Road.

The futuristic design is the company’s second attempt to launch a major development at the site following a massive backlash from residents over the height of a similar scheme in 2009.

Menta withdrew the plans before they came before planning councillors, and have spent the past two years working with the council and residents to create a more palatable plan.

But Addiscombe councillor Sean Fitzsimons said he believed the new application was still too enormous to win over the public.

He said: “This is a better application, but I just don’t think people want a tower that size.

“I’m as geeky as the next person, and when I go up to London I always look at the Shard and all sorts.

“But there’s a difference to the visual impact of that impressing someone coming in on a train, and having to live 50 feet away from it in a Victorian house with people looking into your back
gardens 24 hours a day, seven days a week.”

Cara Bamford from architect Make said: “From the outset we wanted to match Croydon’s ambitions for itself and create an elegant landmark building that offers a lifestyle as totally removed from
outdated concepts of ‘high rise’ living as possible.

“Our building will host a genuine ‘vertical community’, with shared amenity spaces at all levels including roof gardens, games room, craft room and business centre, and rooms available for private
hire for family celebrations and other events.”

More information is available at cherryorchard-croydon.co.uk.

Croydon paedophile scout leader sent to secure mental hospital

4:48pm Monday 4th April 2011

• Print
• Email
• Share
• Comments(0)

A paedophile scout leader who hoarded thousands of child porn movies and pictures on his computer has been sent to a secure mental hospital.

Simon Tapsell, 29, was caught by officers from the Metropolitan Police paedophile unit after spending hours surfing sick websites and
chatrooms for illicit films and images.

Tapsell appeared at Croydon Crown Court today (Monday) for sentencing but a pre-sentence report prepared by the probation service suggested he needed further assessment at a specialist unit.

The paedophile will now spend 12 weeks in the Behavioural Disorder Unit at the Bethlem Royal Hospital in Bromley.

The scout master was suspended by the Scout Association when he was arrested by police after amassing the vile collection.

His secret stash included clips of boys and girls as young as six posing naked, masturbating and having sex with each other and adults.

Officers found 6,571 images and 3,358 illegal movies on his personal computer when they raided the home he shared with his parents in Blackhorse Lane, Addiscombe in August 2009.

One of the most serious involved a boy having sex with a dog.

Tapsell, who suffers from Asperger’s syndrome, pleaded guilty to 18 charges of making and possessing indecent photos of a child at Croydon Crown Court on December 10 last year.

Tapsell is due to appear at Croydon Crown Court again on June 27 for sentencing.

Digital marketing agency Lakestar Media claimed the title of best ‘SEO and PPC agency’ at the 2011 How Do Awards.

The How Do Awards celebrate the media industries best talent, with accolades covering every major marketing and media discipline, from PR and broadcasting, to digital media.

To scoop the award from a shortlist of seven agencies Lakestar Media successfully demonstrated consistently high standards of work including the retention of clients and new client wins. The company showcased award winning campaigns and creative work that has provided exceptional search results and impressive return on investment for it’s for clients.

Neil McKay, CEO of Lakestar Media, said, “We really appreciate winning this award, it’s a real team effort and I am very proud of everyone within the business. As importantly I would very much like to thank our clients for their continued support of our business”

ENDS

Press information: James Crawford, Account Director 0161 235 0316 or james.crawford@citypress.co.uk (issued on behalf of Lakestar Media by Citypress)

Notes to editors:
Lakestar Media is a digital marketing agency based in Manchester and London. Established in 2007, Lakestar delivers a mix of services which include search engine optimisation (SEO), social media, pay per click (PPC), affiliate marketing, display advertising and news feeds for its clients.

Further information on Lakestar Media can be found at www.lakestarmedia.com

If you have picked out to market place your corporation with internet search engine trading key phrase research is ones next palpable step. SEM, (online search engine marketing and advertising), is a great way to get your web business started. The way you will need to begin is with a great keyword search tool to help discover good keywords to utilize.

What are you marketing and / or delivering being a company? Try to believe just like a consumer who is searching for this distinct thing. Put in writing how you should search for it on-the-web. Here will be the setting up stage for your keyword and key phrase research.

Type these ideas, one at the same time, straight into the keyword finder of ones choice. The key word finder can then be ready to give you even more advice of any other search terms will possibly not have considered. Wide open a Notepad file and save these key terms for later apply.

If you need a leg up, try to get out what keyword phrases ones competing firms are using through locating the key word tags on their web site and then right visiting in a blank area of the website, then visiting on ‘view source’ in the drop down menu. Refine those keywords simply by making them a significantly more accurate information what it’s always you are retailing or providing as being a service and implement them to your benefit as one online search engine marketing and advertising keyword.

It’ll take a little time, maybe even several months for you to receive a deal with on what key phrases work best for you and those that will not. Whenever you have to this level, eradicate those which fail and replace them with a new one then repeat the procedure. Keeping your keywords recent will guarantee your website stays at the ultimate of the online search engine results website page.

Some other tips to follow when it comes to utilizing keywords properly contain working with them in content articles and blog-posts. In articles or blog posts, be certain you do not implement the key word you’ve got picked out a lot more than three times. When in the first part, on one occasion within the body of the content article and once within the last few section of the content. Ones content articles ought to be at least 500 thoughts.

Blog articles tend not to need to be that long, typically three hundred fifty words and your search term should just be used 3 times, in the name, when in the initial sentence or two and once over the last sentences.

Irrespective of whether you’re posting articles or blog posts or blog posts be careful that each and every one is original content material. No identical article content authorized. All that means is, when you write two and / or more articles, they cannot say the distinct very same thing in the precise similar way. Mix it up. Implement different phrases that signify the identical thing. Time for you to draw out that imagination.

One some other thing, ones website name. You will get ranked increased if your website address is ones search phrase. Makes meaning doesn’t it. If your domain name is ones keyword and key phrase and someone lookups for that accurate keyword and key phrase they will certainly see your web site listed first then go to your web site to buy what they already want. Easy.

All the things is related, you start with your domain address right on down to your blog articles and your picked out online search engine sales search phrase.

DSM Publishing is a reliable SEO agency that offers variety of SEO services to help you with your SEO campaign needs.

There’s an old story of two hikers who come across a hungry-looking bear in their path. The first hiker starts running for his life. The second hiker stops, sits down on a rock, takes off his heavy hiking boots off, reaches into his backpack and pulls out a pair of running shoes.

The first hiker yells back at him, “What the heck are you doing? You know you can’t outrun a bear!” To which the second hiker matter-of-factly responds, “I don’t have to outrun the bear, I just have to outrun you.”

To me, running PPC campaigns is a lot like being chased by a hungry bear twenty-four hours a day, seven days a week. Most of the time, I feel like I am running flat out just to keep even with competitors and to keep traffic and conversions flowing profitably from our campaigns.

Today, I’d like to take a few moments to pause, sit on a rock, and think like that second hiker.

What can we do to leave our competitors in the dust once and for all rather than have them nipping at our heels all the time? What does it take to create real, sustainable long-term advantage in competitive PPC markets?

Short Term Advantages Disappear Quickly

The first thing we all have to realize that nothing inside our campaigns today gives us any long-lasting advantage.

No matter how big our keyword lists, how clever our ads, how scientifically we set our bids and how attractively laid out our landing pages are, it doesn’t take long before competitors can see exactly what we are doing.

Think your great keyword inventory gives you a sustainable advantage? Fuggedaboudit!

Keyword research used to be one of the secret high-arts of paid search magicians. You could sift through some keyword tool reports, create long keyword lists, write a few ads and then sit back to enjoy all the traffic from your campaigns. Not any more.

Now, keyword research is so easy, even kids can do it. As a matter of fact, as I look around at search conferences lately, it is clear that kids are doing it! Well, okay, maybe it’s just me getting older. (Note to self: time to update author bio photo.)

Google and Microsoft offer amazing keyword suggestion tools for everyone. And no matter how clever your keyword research, or how well you guard it, it does not take long for competitors to discover what words you are bidding on and how much you are spending by subscribing to tools like Adgooroo, SpyFu, Keyword Competitor, and SEMRush, to name just a few.

Think your great ad copy is a sustainable advantage? Think again. The minute your ads go live, your competitors can see them and react to them with ideas, counteroffers and calls-to-action copied right from your ads. Your advantage may last less than a day.

Think your new landing pages give you a sustainable advantage? Why?

Landing pages are also public and available for anyone to see – just one click away from the new ads you put online. Maybe it takes a few days or few weeks for aggressive competitors to redesign landing pages of their own, but that’s about it.

On a relative scale, here is how long these advantages will last:

Tactical Changes vs. Strategic Improvements

Don’t get me wrong. All of the campaign changes you make, day-in and day-out, adding keywords, writing ads, adjusting bids, and launching new landing pages, and are highly important. However, they are also very tactical and only impact short-term performance.

I have seen account managers in many companies, (myself included) make changes one day, forget what they did and why they did it, and reverse the changes a week later.

The problem is that making changes is really, really, easy to do and gives you such a power feeling of accomplishment. Spur of the moment inspirations are as likely to hurt performance as much as improve it.

Here are some common examples:

• Accepting AdWords optimization suggestions and letting AdWords decide what ad group they belong in.
• Changing match type from phrase to broad to increase traffic, or changing match-type from broad to phrase match or exact because you CPA was too high yesterday.
• Increasing search keyword bid to get more traffic without regard for ad position.

If you are spending most of your time and energy tweaking things this way, you will always have hungry bears breathing down your neck.

Your Only True Sustainable Advantage: Better Process

Okay, so I’ve made the point that any competitive advantage you enjoy today will go away as quickly as your competitors see what you’ve done. It may take a few days, weeks or months, but sooner or later, your competitors will try to match you step for step.

This is especially true if you are the recognized leader in your marketplace.What your competitors can not emulate are the thought processes that drive your decision-making. The better your decision-making process and the faster you can implement improvements, the harder it becomes for competitors to keep up with you.

So what constitutes better process?

Better process starts with a mindset that you want to get better every day, regardless of what your competitors are doing. Rather than looking over your shoulder to see what hungry competitors are on your tail, focus instead on your own company’s unique challenges and work hard to improve yourself. I guarantee you there is enough work there to keep you busy for years to come.

Better process requires improving your ability to test, measure, report and analyze your campaigns so you can generate deeper, more valuable insights and more granular, higher-quality information.

We all have plenty of data to work with; what separates winners from losers are the strategic insights they generate from their testing and reporting systems. There is no more powerful competitor than one who takes the time to understand their data, and by extension, their customers better.

None of us is born with the innate ability to design, implement and measure PPC test results. And even though we even know how important testing is in PPC advertising, few of us have formal training in statistics and experimental design.

Sure, we can rattle off expressions like A/B split testing, multivariate testing and the Taguchi method, but do we really understand what they mean? What does it mean to design a test, establish a good control and isolate variables.

We all have heard the expression “wait until you have statistically valid results” but would any of us recognize a valid result if it walked up to us and shook our hand? What is a confidence level, anyway? What is an F-test?

No matter what you know or don’t know now about testing, right now, start now to set aside time for learning. Dust off that old stats book and get ahold of books and blogs dealing with experimental design, testing and measurement.

In the short term, of course, you’ll have less time to mess around inside your PPC campaigns, but in the long run, you will develop a solid foundation for campaign optimization that will keep you miles ahead of your hungriest competitors.

Opinions expressed in the article are those of the guest author and not necessarily Search Engine Land.

Related Topics: How To | Intermediate | Paid Search