De-portalization and Internet revenues

This post is a little more philosophical than most that you will see here. It provides a little bit of background as to why edgeio is in the business of bringing together, organizing and distributing listings to the edge of the network. In short it is because we believe that the Internet is moving away from big centralized portals, which have gathered the lions share of Internet traffic, towards a pattern where traffic is generally much flatter. The mountains, if you will, continue to exist. But the foothills advance and take up more of the overall pie. Fred Wilson had a post earlier this week about the de-portalization of the Internet which is essentially making the same point when seen from the point of view of Yahoo.


Update: 11am Pacific, Sunday 10 December

Several commentators are seeing the word “de-portalization” (first coined by Fred Wilson) and reading “end of portals”. To be clear, and apologies if I wasn’t already, de-portalization represents a change in the relative weight of portals in a traffic sense, and the emergence of what I call the “foothills” as a major source of traffic. This will affect money flows. Portals will remain both large and will continue to grow. But relativeley less than the traffic in the foothills. The foothills will monetize under greater control of its publishers and the dollar value of its traffic is already large and will get much larger.


The following 3 graphics illustrate what we believe has happened already and is likely to continue.

The first picture is a rough depiction of Internet traffic before the flattening

2004 and all that

The second picture is a rough depiction of today – with the mountains still evident, but much less so

The rise of the foothills

The third picture is where these trends are leading. To a flatter world of more evenly disributed traffic.

The future pattern of web traffic

Some of the consequences of this trend are profound. Here are our top 10 things to watch as de-portalization continues..

1. The revenue growth that has characterized the Internet since 1994 will continue. But more and more of the revenue will be made in the foothills, not the mountains.
2. If the major destination sites want to participate in it they will need to find a way to be involved in the traffic that inhabits the foothills.
3. Widgets are a symptom of this need to embed yourself in the distributed traffic of the foothills.
4. Portals that try to widgetize the foothills will do less well than those who truly embrace distributed content, but better than those who ignore the trends.
5. Every pair of eyeballs in the foothills will have many competing advertisers looking to connect with them. Publishers will benefit from this.
6. Because of this competition the dollar value of the traffic that is in the foothills will be (already is) vastly more than a generic ad platform like Google Adsense or Yahoo’s Panama can realize. Techcrunch ($180,000 last month according to the SF Chronicle) is an example of how much more money a publisher who sells advertising and listings to target advertisers can make than when in the hands of an advertiser focused middleman like Google.
7. Publisher driven revenue models will increasingly replace middlemen. There will be no successful advertiser driven models in the foothills, only publisher centric models. Successful platform vendors will put the publisher at the center of the world in a sellers market for eyeballs. There will be more publishers able to make $180,000 a month.
8. Portals will need to evolve into platform companies in order to participate in a huge growth of Internet revenues. Service to publishers will be a huge part of this. Otherwise they will end up like Infospace, or maybe Infoseek. Relics of the past.
9. Search however will become more important as content becomes more distributed. Yet it will command less and less a proportion of the growing Internet traffic.
10. Smart companies will (a) help content find traffic by enabling its distribution. (b) help users find content that is widely dispersed by providing great search. (c) help the publishers in the rising foothills maximize the value of their publications.

edgeio is hoping to play a role in these trends. We will talk about some new products later in the month that follow from this approach.

Discussion

Kevin Burton
Techmeme
Mike Arrington
Syntagma
Keith Teare’s Weblog
Dan Farber at ZDNet
Mark Evans
Fred Wilson
Ivan Pope at Snipperoo
Tech Tailrank
Collaborative Thinking
David Black
Surfing the Chaos
Ben Griffiths
Dave Winer (great pics)
Kosso’s Braingarden
Dizzy Thinks
Mark Evans

We’re relevant! Search engine launched at edgeio

Hot on the heels of our acquisition of Adaptive Real Estate Services we can today announce the initial roll out of edgeio’s relevance based search engine. This is the first step in our efforts to make edgeio.com the best place to find “stuff” anywhere in the world.

By way of background, edgeio launched in March with zero listings. We took in about 100 new listings per day at that time. Today we take in about 700,000 new listings per day. The search engine we began with (free text matching and then results in reverse chronological order) simply was not good enough to function with this number of listings.

We now have a dedicated search team and this is their first push. It is not yet perfect but it is a vast improvement on what was there before.

In this upgrade we are acknowledging the way partners and users are using edgeio and trying to improve their experience. Many listings based sites are uploading their listings to us and we are providing search traffic back to them. We are being used as a listings search service by companies with listings and by users looking for listings. A “search engine for stuff” if you will.

Here are a few searches to try:

Used iPod
Sony Vaio
Baby Crib

These are all global searches (edgeio has data from about 15000 cities worldwide). You can use the geography widget (top right of the results screen) to choose a city. Once you have done that then the slider control can be used to fine tune the results (zip, city, state, country, continent, world). Of course, you can also sort by price or by date listed.

Arun Jagota; Josh Myer and Dale Johnson are the team – mostly quite new at edgeio – who are working on search, and have moved us from a reverse chronological display of results into a relevance ranked display. Of course they have had a lot of help from others, most notably our technical advisors. And they have a lot of work still to do to make the results the best there is.

Going forward, as edgeio strives to bring together, organize and distribute the world’s marketplaces, edgeio.com will be the place that our organizing efforts are most obvious. It will be the place to find “stuff”.

From here on relevance will be our default sorting method. Of course we will enable users to modify the sort order (by time, by price, and in the future by other criteria). Our outbound APIs will eventually reflect these options also.

There is a whole lot more to come from us, and this is a baby step in many ways, but a significant directional move. Let us know what you think.In future posts we will talk about the bring together and distribute parts of our vision – these are realized through our edgedirect product.

But for now lets meet the team working on search:

Arun Jagota

Arun Jagota I am a search engineer at edgeio. I am working on the design and evaluation of algorithms for improving relevance in particular and search in general at edgeio.

One of the key challenges is the relevance problem itself. A tough nut to crack. The challenge is to find methods that are both simple and efficient, yet effective in returning relevant results. Another challenge (specific to edgeio) is to fetch relevant results from a variety of sources in real-time, recompute their relevance internally in real-time, and merge them into a single set of results that the user sees. A third issue, also specific to edgeio, is that our documents (unlike general web pages) are listings in verticals with varying degrees of structure. So there are special issues involving relevance and search for finding “stuff” rather than web pages.

What keeps me motivated is that “relevance and search” supply me with a constant source of challenging (but not impossible) problems to solve, and algorithms from computer science, statistics, and information retrieval present me with solution methods to consider and evaluate. Another thing that keeps me going is constant incremental progress and quick feedback. You have an idea, try it out, sometimes it improves relevance, and you notice it quickly.

Before working at edgeio, I worked at another start-up (Xoom corporation) as a data analyst and machine learner. There I designed improved algorithms for predictive modeling in an e-commerce setting and also some for improved fuzzy matching of names and addresses of people. Prior to that I taught graduate courses as an adjunct faculty member in computer engineering at Santa Clara University, including one on “Information Retrieval And Search Algorithms”.

Josh Myer

Josh MyerHi, I’m Josh. I’m the Young Guy at the office, but I make up for it with an intense background. Before going to college, I spent a few years working as a reverse engineer and general puzzle-solver, in fields ranging from accounting to instant messaging. I just wrapped up two degrees from the University of North Carolina (Chapel Hill), one in Linguistics and one in Mathematics. I focused on the typically-impractical formal aspects of both, but it’s actually come in handy when working on search problems.

I spend a lot of time in the plumbing of edgeio, but have been working more on search lately. The user-visible bits that I’ve done so far are the real-time search results from external providers. I’m currently working on several things to make search better, faster, and more user-friendly.

Working here has been great: there’s always a new problem to solve and the freedom to solve it the way you want to. All told, I get to use my entire background at work, ranging from unix arcana to the acquisition of language in children. It’s all the fun parts of college (laid-back, lots of new knowledge) with the fun parts of a job (making useful things, getting paid).

Dale Johnson

Dale Johnson

I am variously “search engineer”, “sphinx developer”, “data platform engineer”, “senior database engineer”, roughly in that order.

I have done 15 years of database work, on relational database, data warehousing and search. I have done work on PostgreSQL internals, and have studied MySQL internals. Most recently I worked at Tellme where I designed and developed a 1.5TB data repository to drive data warehouse reporting functions for the call details of each of over 1 billion Tellme-answered phone calls. This involved a redundant and reliable cluster of over 50 mysql servers using inexpensive off-the-shelf hardware. This used a combination of mysql and record-oriented raw data files.

I am currently coding extensions in C++ to our search engine, Dale's Whiteboarddoing things like parsing out Chinese sentences into searchable blocks to support http://mulu100.com. Also I recently have implemented some statistical approaches to our full text search, gathering a corpus profile and applying that in real time to search terms to improve the selectivity of results.

The key challenge I think is be able to flexible enough to implement a solution as we discover the most natural way for a user to navigate through millions of items. To provide a back-end that is able to support a dynamic state-of-the art interactive user experience that people now expect; and to be able to provide these results in real time. Many requests need to distinguish between tens of thousands of documents which have one or more of the search terms present, and determine the top 10 / 100 / 1000 of those items in under a quarter of a second. Under these operational constraints, the traditional relation database approach completely falls down; quite the fun engineering challenge.

What keeps me motivated is the knowledge that the web is still 95% noise and 5% signal. Search is the thing that has the potential to cut through the noise, so we’re really fighting the good fight, of taking listings from potentially obscure but highly useful sites, and making them available to the people it will really matter to, and doing it in a fair and egalitarian way.

NB. Our email addresses are first name at edgeio dotcom

edgeio acquires assets of Adaptive Real Estate Services

I am pleased to announce that edgeio has made its first acquisition. About a week ago we acquired the assets of Adaptive Real Estate Services, a company built over the past several years by father and son team Robert and Peter Meyer.

ARES, as it is known, has patiently built up relationships with Brokers and Agents in 70 of the top MLS organizations (multiple listings services) in the US. It has about 1.5m homes listed for sale in the areas it covers.

As we integrate ARES into the edgeio platform it will be possible for users of edgeio.com, and partners of edgedirect to show these 1.5 million homes in search results.

The key advance ARES made is that it automates the inclusion of listings via the IDX protocol used by MLS organizations. This is painstaking work as few MLS organizations share common data structures with others. ARES also provides automated web sites for brokers and agents wanting them.

Peter and Robert will introduce themselves on this site over the coming weeks. For now we are pleased to have them on the team and we look forward to helping them serve home sellers and buyers by turning the real estate listings service in edgeio into a major asset for each.

Our goal is “to bring together, organize and distribute the world’s marketplaces”. As part of this we want edgeio.com to become a universal search engine for “stuff”. As that evolves the edgeio platform can become a major marketing vehicle for Realtors. We will be talking more about our goals at the GMAC event in Florida in February 2007.

In the meantime, if you are a broker, an MLS, an agent or indeed a home buyer or seller, please feel free to email Robert Meyer for more information about how he can help you. He is [Robert at edgeio dot com].

Discussion

Techmeme
Dave Winer
VentureBeat
John Furrier
Seattle Post and Intellegencer
Transparent Real Estate
Future of Real Estate Marketing
Realty Thoughts

Welcome to Techmeme Readers

edgeio is advertising on Techmeme for a month from November 27th. So, if you are a Techmeme reader and new to edgeio thanks for coming to check us out. Click here to read this initial post in full, or here to subscribe to our RSS feed.

edgeio is a new kind of search engine. Instead of indexing web pages we index actual items or listings from web sites that are in one of the many listings based businesses. These may be job sites, real estate sites, sites for buying and selling cars, classified listings sites, sites like eBay and Amazon which sell general merchandise, and even blogs where the owner lists items using the tag “listing”. Sites with listings submit their listings to edgeio (we don’t scrape or crawl listings unless the publisher requests that we take them). Since our site launched in March of this year we have reached some great milestones.

– We have gone from zero to over 100 million live listings
– We have gone from listings in 1 city to listings in over 15,000 cities
– We have launched a Chinese language web site at http://www.mulu100.com
– We have integrated real time search from Amazon, eBay and CafePress into our edgeio.com search engine.
– We have changed from a founder funded company into one with over $5m of angel and venture funding.
– We have grown our team from 1 engineer, 1 product manager and 2 founders to over 20 staff (mainly engineers, some in Beijing, China).

Our vision is to bring together, organize and distribute the worlds marketplaces . This will take a few more months, and probably even years, of execution but we are very clear about the goal. To that end we want to turn edgeio.com into the best place to find “stuff” from our listing partners (jobs, cars, homes, books, bicycles, babysitters, friends) anywhere in the world. We want edgedirect to become an easy to use platform for partners of all types to get their “stuff” into the edgeio engine and so benefit from the traffic we and our distributors can send back to them. We want to provide many means for third parties to become edgeio distribution partners, taking listings and earning revenue through re-publishing them.

During the next 2 months we will be rolling our several new services. Pretty shortly edgeio.com will move from having a time based search ranking system (last in show first) to a pure relevance based system. Secondly, we will launch edgeio marketplaces in beta (more details to come at the appropriate time). We will develop more ways for listings sites to use edgedirect to upload their stuff into edgeio. We will also enable a greater range of methods for third parties to take listings from edgeio and re-publish them.

During our month on Techmeme we will use this blog to introduce you to some of these new services, and to introduce the people behind the scenes who are now part of the edgeio team.

The next post (later this week) will be news of our first acquisition.

Facebook launches Social Bookmarking, edgeio participates

Last night Facebook announced that it has launched “Share”, a social bookmarking infrastructure. Blog post here. Reuters coverage here

edgeio is a launch partner for share. You will notice that there are some new elements in two places.

Firstly on the search results page at edgeio.com.

Search Results with Share

And secondly on the item detail page

item detail with share

In both cases we have also added del.icio.us; digg and wink buttons.

This should be a great tool if you are looking for something and find multiple options. You can save the ones you are interested in and then use the saved items as your shortlist of things you are interested in.

Let us know if you like it.

feedback can be sent here

Links:

Venturebeat – here
Mashable – here
Inside Facebook – here

Series A financing, China web site and Patent filing

This is the text of a press release we are issuing today (Tuesday 24th October 2006)

Menlo Park, California: October 24th 2006

edgeio corporation announces Series A financing, Chinese web site and Patent filing.

edgeio corporation, a leader in online listings announced today that it has raised $5 million in the first close of a Series A round of financing. The round was led by Intel Capital and also included an investment from Transcosmos Investments and Business Development Inc, a Japanese public company with a Silicon Valley investment arm focused on Internet-based U.S. technology companies expanding into the Japanese marketplace. Investors in the company’s angel funding had their investment converted into A round stock. These include Ron Conway, Jeff Clavier, Claudio Chiuchiarelli, Frank Caufield Jr., RSS Investors, Millenium Technology Ventures, Michael Tanne, Auren Hoffman, Sam Perry, Bill McCabe and Louis Monier.

Since its launch in March 2006 edgeio has successfully become a significant player in the rapidly growing infrastructure for online listings. It has become an essential service to many vendors, across many verticals in many countries, who use it as a part of their strategy for distributing listings and catalogs to attract more customers.

edgeio has over 3000 publishers who upload their listings via its edgedirect service. These include key players in the jobs, autos, real estate, leisure and other sectors. Many bloggers are publishing listings to their blog and having them automatically entered into edgeio through the use of the “listing tag�?. edgeio is also a featured distribution channel for many sites offering to help distribute listings, such as VFlyer, Postlets, the Point2 network and others. Beyond distributing uploaded listings, edgeio brings together content in real time from partners such as eBay, Amazon and CafePress, creating the world’s largest search engine for listings at edgeio.com.

“Our goal is to bring together all of the world’s marketplaces, to organize them vertically and geographically, and then to make this enormous inventory of catalogs available, through a single interface, to those who would like to take these listings and create new online marketplaces,”

said Keith Teare, Chief Executive Officer and founder at edgeio.

Since launching six months ago edgeio now has more than 100 million listings in many verticals, (about 700,000 new each day). These come from over 14,000 cities in 130 countries. By being an aggregator of such a large number of listings edgeio.com can become a serious search engine for jobs, homes, cars and many other items, and can send significant amounts of traffic back to the original publisher.

The company also announced today that it has launched a Chinese language version of its web site named mulu100.com (which in Chinese means catalog of catalogs). The Chinese service has initially formed a partnership with edeng.cn, a China based listings site, similar in many ways to craigslist.com.

“Chinese listings began to appear on edgeio within a few days of us launching. And we saw Chinese searches on our data grow significantly during the summer,”

said Keith Teare.

“We decided that the Chinese market would be our first localized site and we now have a significant number of listings from all over the country. That is why we have launched mulu100.com as a fully Chinese front end to the edgeio data. edeng has significantly expanded our ability to include China based listings and proven to be a great initial partner.”

edgeio corporation also announced that it has filed for patents covering its distributed marketplaces architecture and many of the features of edgeio.com. The patent claims were provisionally filed in October 2005 and have now been completed as full filings.

About edgeio corporation:

edgeio is a a leader in online listings. Its vision is to bring together, organize and distribute the world’s marketplaces. Products include edgedirect, edgeio.com and mulu100.com. edgeio is based in Menlo Park, California and was founded in 2005 by Keith Teare, its Chairman of the Board and Chief Executive Officer, and Michael Arrington, a member of its board of directors. For more information see http://www.edgeio.com.

About Intel Capital:

Intel Capital, Intel’s venture capital organization, makes equity investments in innovative technology start-ups and companies worldwide. Intel Capital invests in a broad range of companies offering hardware, software and services targeting enterprise, home, mobility, health, consumer Internet and semiconductor manufacturing. Since 1991, Intel Capital has invested more than US$6 billion in nearly 1,000 companies in more than 40 countries. In that time, about 180 portfolio companies have been acquired by other companies and another 155 have gone public on various exchanges around the world. In 2005, Intel Capital invested about US$265 million in about 140 deals with approximately 60 percent of funds invested outside the United States. For more information on Intel Capital and its differentiated advantages, visit www.intelcapital.com.

About Transcosmos Investments & Business Development, Inc.

Transcosmos Investments & Business Development, Inc., is a wholly owned subsidiary of Transcosmos, Inc. With offices in Bellevue, Wash., and Mountain View, Calif., Transcosmos Investments & Business Development is a strategic corporate investor and an active local business partner for Internet-based U.S. technology companies expanding into the Japanese marketplace. It is the international partner-of-choice, offering capitalization of joint ventures, infrastructure development, strategic relationships, staffing and localization to companies in the U.S. technology sector – effectively creating a pipeline of technological information to Japan. Transcosmos Investments’ portfolios, which focus on companies specializing in digital marketing, e-commerce and digital media, include such marquee brand names as AskJeeves, Inc./Bloglines, DoubleClick Inc., RealNetworks Inc., NetRatings Inc., Atom Shockwave, Inc., Become Inc. and CinemaNow, Inc. For more information, please visit: http://www.transcosmos.com.