Sorry for the lack of posts recently, but we’ve been busy changing and improving Parse.ly for the better!

We did, though, get picked up by a couple popular blogs in the past few weeks. Here a few snippets from both ReadWriteWeb and ZDNet.

Bloggers, muckrakers and news fanatics, lend me your ears. It’s entirely possible that we’ve discovered one of the best approaches to media monitoring since RSS itself. My mother always said, “You’ll never get what you want unless you ask.” But with adaptive feed application Parse.ly, that simply isn’t true. Rather than forcing us to abandon our overflowing feed readers, Parse.ly records our preferences and learns to work with us.

ReadWriteWeb 11/11/09

I haven’t figured out a way to manage Google Reader. I tried using Fever, but it doesn’t find news that matters to me… and it cost $30.  Techmeme is my home page, but I think it needs an upgrade.  I would like a feed reader that saves favorite feeds for me, and finds other content that is similar and interesting.

A new product called Parse.ly caught my eye that makes content discovery a painless process.

ZDNet – The Web Life 11/24/09

Check out our press page for more articles written about Parse.ly.  We’ll update you soon about what we have in store for the future!

Hi Parse.ly fans.  Andrew here.  I just wanted to let you know that I presented Parse.ly at the NYC Search & Discovery Meetup on Thurs, Oct. 29.  The meetup is organized by Otis Gospodnetic (blog), who is one of the authors of Lucene in Action and the author of the forthcoming Solr in Action book.  It was graciously hosted at kgbweb (thanks for making that happen, Joe West!).

We make heavy use of Lucene and Solr on Parse.ly, so it was exciting to get an opportunity to present to a community of fellow technologists building systems with these excellent technologies.

Here is the abstract from the talk:

Parse.ly: Inside a modern RIA built with Solr
Andrew Montalenti
—
Parse.ly is a rich, adaptive web application that discovers your unique interests to filter and prioritize content from countless news and blog sources on the web. This talk will introduce Parse.ly with a quick demo and then delve right into how the Parse.ly engineering team makes use of the Solr open source search engine. This will include discussion of initial design mistakes that were later revised and “real world issues” that were overcome in scaling a system that currently processes millions of articles per week. Finally, we will discuss the existing Solr and Python landscape, and how we at Parse.ly aim to help the Solr community with the open source release of high-quality, Pythonic components for doing common Solr tasks.

Otis has written about the talk, and the slides are online, as well.  Special thanks to my kickass Parse.ly colleague Didier for setting up our BitBucket repository and starting to tease the code out that is ready for the community.

Thanks also to everyone who attended, and if you have any questions about it, feel free to contact us.

Andrew’s a lifetime New Yorker. He grew up in the city and Long Island (and now lives in Astoria, Queens), and has always been a fan of the New York. I’ve been in NYC for the past eight years bouncing between Mnhattan and Brooklyn (I just moved to Ft. Greene this past weekend), and have grown to love the city

Well, this past week we had the opportunity — the privilege — to present Parse.ly (our baby) at NYC Demo Day, an event focused exclusively on New York City, early-stage startups.  You can check out some photos of Andrew and I presenting here.

The event was a blast, as evidenced by Andrew’s inexplicably happy face after it was over (see the photos to see what I mean).  Good luck to all the companies: Renthop, Trendsta, SECWatch, LegalRiver, Sensobi, Localytics, Seatgeek, Postling … and, of course, Parse.ly!

If you were trying to log into Parse.ly between 11pm-1am this Sunday, you may have noticed that it was intermittently down for maintenance.  Over the last several weeks, we’ve been working hard to roll out some new features, polish some rough edges, and improve our infrastructure after our launch last month.  Our first beta users have been amazing in providing us with detailed and specific feedback on what works and doesn’t work well within Parse.ly.  We’ve diligently addressed many of the issues raised by these users and rolled out a new version of Parse.ly this weekend.

So, what’s new in Parse.ly?

• Faster archiving and deleting: many users have complained that archiving and deleting items in Parse.ly was extremely slow.  This was a shortcoming of the beta release we pushed out in August, and we have now rectified this issue.  Archiving and deleting should be very speedy, and bulk archives/deletes are just as fast as single ones now.
• Keyboard shortcuts: we bill Parse.ly as the most productive way to read the news, but our initial beta users noticed that one productivity feature we are sorely lacking is keyboard shortcuts.  We have implemented an experimental cut of keyboard shortcuts in the latest release.  Simply click inside the article grid (under “Current Items”) and press the question mark key (?).  This will bring up a message box with a list of our keyboard shortcuts.  They are very similar to GMail’s; for example, e archives an item, and shift+i marks an item as read.
• Username casing problems: we had a silly bug where your username was case-sensitive, even though we use e-mail addresses for usernames.  This has been addressed and so now you can use any case you like for your username.
• Better De-duplication: we did include a deduplicator in our initial beta release, and we thought it was working pretty well: nearly 50% of the articles processed by our crawler were being deduped.  However, many users reported duplicates in their accounts, and when we investigated, we saw that there were many scenarios that fooled our deduper and resulted in an unpleasant reading experience.  In particular, resyndicated articles (e.g., those republished after a press release) often had similar titles/summaries but were not exactly the same.  So we have included a more aggressive deduper that makes use of fuzzy string matching and seems to work better in the testing we have done so far.
• Better sources: we are continuing to improve the breadth and depth of Parse.ly’s sources.  We have restructured our crawler so that it gets access to sources that weren’t available before, and also better delivers content based upon your interests.
• More frequent crawls: many users were reporting that crawls wouldn’t happen often enough for them, so they wouldn’t see articles published e.g. in their twitter feed right away in Parse.ly.  We have re-engineered our crawl infrastructure to be much more performant, which will mean fresher content inside Parse.ly for you.
• You can kill the Parse.ly share bar: when you click a link inside Parse.ly, we top-frame it to allow you to share it easily with your friends.  However, users rightfully complained that they wanted the ability to easily remove this bar from any page.  You spoke, we listened!
• Password recovery: some users forgot their passwords and we simply forgot to implement a password recovery feature in our initial release.  It’s there now.
• User profile editing: users can now change their password from inside the UI.  This was an obvious feature just plain missing from the initial release.

We also made some improvements to our Interest Setup Wizard, but this will mostly affect users who first sign up for the system.  (You did remember to invite your friends to join Parse.ly, right? )  These include:

• Better performance when adding new interests: before, there was a bit of a lag when you entered in interests.  This should be fixed now.
• E-mail is sent to user upon initial setup: some of our users were rightfully confused when they entered in their interests and … nothing happened.  We aren’t activating accounts right away, but at the very least, we should send you an e-mail to let you know that your interest entry actually worked!  This is fixed now.

We are also working on some big changes within the Parse.ly engineering team to take our product to the next level.  We have partnered with our excellent hosting company, The New York NOC (NYNoc), to scale out our infrastructure and process more content than ever before.  We are also planning our future iterations where we hope to innovate and deliver more features to save you time and let you discover the content that best matches your interests.

A Personal Note from Andrew

As the lead developer for Parse.ly, I just want to say “Thanks!” to all our awesome users.  The thing that has impressed me most about Parse.ly’s users so far is how detailed, intelligent and thoughtful their feedback has been.  We want our company to be driven by your feedback, so please, do not hesitate to provide more of it on our Cog Tree Get Satisfaction Page, on Twitter, or directly via e-mail.

This last few weeks reminds me of something Jim Young, the founder of HotOrNot.com, said to us at DreamIt this summer: “when the site became successful, we were left having to figure out how to change the engine while the car was still running.”  It’s going to be a lot of work evolving Parse.ly even while providing the service to our existing users, but I’m looking forward to the challenge.

Thanks again to all our users for their feedback so far, and I hope you enjoy the new version of Parse.ly!  (Also, don’t forget to report any bugs or tell us what is working well for you.)  And if you haven’t signed up for Parse.ly yet, what are you waiting for?  Do it now at http://parse.ly

Wow, it is hard to believe that this summer is almost over.  It seems like Cog Tree just moved to Philadelphia for the summer last week!  But a lot has happened and we’re really proud of what we created.  Let me try and breakdown the summer to give you an idea of how quickly we moved:

May 23rd – Moved to Philadelphia to participate in DreamIt Ventures’ second program for the summer.

June 1st – We knew that information overload was a serious problem, and that innovative technologies could be applied to help manage content online.  Though we had an idea of WHO it was affecting the most, we weren’t quite sure about HOW this was specifically affecting companies and people.  So, with the problem simmering in our heads, we went straight to the market to figure out what pain points existed and what solutions were available.  On the technology end, we had to bring a team together and start building out the prototype.

July 1st – Had an initial wireframe that was constructed directly from market feedback and major infrastructure work complete.

July 1st to August 1st – We went back to the market to ask questions about how our free tool could solve problems beyond any other free tool available.  And also asked about what demand there was for specific features that would be built on top of Parse.ly.  From this user feedback, we built Parse.ly.

August 13th – DemoDay 2009.

DreamIt Ventures was an outstanding program that allowed us to really dig deep into a problem that has persisted for the past few years.  We know that we have a solution that will save our users time and money.

Sign up today and we’ll provision you with an account as soon as possible.

In their recent posts, Shabadu and UB have a discussion about whether limiting who can post content on the internet is a good or bad idea. Here’s my response.

Shabadu:

You can’t limit the internet! Limiting the internet is like putting a leash on Lassie when she is out there trying to save Timmy. The internet, for all its ridiculous memes, articles, pictures, and videos, has created innovation through its low cost, its efficiency, its depth, and, most importantly, its freedom. If you limit what people can put on the internet, then you are, in effect, limiting the way that the internet can morph itself and change the things that are connected to it. That is bad.

UB:

On the other hand, Shabadu has a point. The internet has evolved in many ways. The combination of cheap computing power, expensive services moving to the cloud, and more advanced web technologies allowed the internet to blur the line between a website and a desktop application. But that’s not the only thing that’s changed. Right now, it seems that web technologies are taking a backseat to content. Internet content has the spotlight right now; though it seems like it may have cold feet.

I think people are waiting for the internet to solve all content problems. The slow collapse of traditional (filtered) media is causing people to look towards the internet to become the main content provider. And I would say, for a decent amount of people, it has become the number one source of content.

This is where Shabadu’s argument comes in to play though. Shabadu says:

So, no, the average person shouldn’t be able to create their own websites or edit their own videos or write their own stupid blogs (Note: I do understand the definition of “irony”) just because they can afford broadband internet access. These “creative types” would be better served helping actually talented people further their endeavors instead of filling the world with mediocrity. For every legitimately funny person, there are thousands of idiots with too much free time on their hands who think they’re accomplishing something of value when all they are doing is making it harder for me to find a clip of a guy getting hit in the nuts.

And he’s right. This is a big problem. Anyone can blog, Twitter, Facebook, FriendFeed, LinkedIn, Wordpress, Tumblr, Flickr, Vimeo, YouTube, Delicious, Digg, Reddit, Mixx, Blogspot, Typebad, Blogger… I think you get the point (yes, I used them all as verbs). This is a too much though. On the other hand, if the internet was limited to pristine content such as the NYTimes OR to highly visible blogs/websites like CollegeHumor, then it would miss a huge section of information that people are inhaling as we speak.

Here’s the question: How much value do you lose if you ignore the Social Media aspect of the internet? The answer is very unclear and this is where internet content is getting coldfeet. The internet needs to convince the masses that there is value in the firehose of Social Media that is out there. I don’t think it has done this yet though. Sure some of enterprise has taken to it; but the majority of people are just not engaged in it. And when they become introduced, will they embrace, or become frustrated like Shabadu?

The verdict on internet content is still out. Will it prove its value over all other mediums or will there be a backlash to the massive amount of information?

In conclusion, Shabadu, and UB, you are both wrong.

[I'm not usually this mean, but Shabadu deserves it, sorry UB.]

Drawing By: Eric Drooker

The fact that we’re inundated with information consistently needs no explanation, and is not something that has recently evolved.  Information has been thrown at me my entire life, through print, radio, TV and now the Internet.  However, before the Internet, the way that information hit me was a little different.  News through TV programs and articles was less dependent upon what source I picked, and more dependent upon the general happenings of the day.  There were a limited amount of sources and they all were trying to accomplish one goal: inform the viewers of the news.  As a consequence, news was roughly the same.  By this I mean that news programs had to compete with each other through a similar set of values.  They had to have tough reporters who were looking to grab the most breaking news.  They had to provide consistently balanced content that strove to be objective and honest.  They were all measured by a core set of principles that were a necessity to exist within the domain.

Not all sources were equal though.  For instance, CNN was able to grab a significant audience through their excellent reporting of the Gulf War.  NYTimes, Washington Post, Chicago Tribune and like newspapers were able to use their vast resources to keep tabs on the local, state and federal government as well as international reporting.  All sources though, more or less, adhered to the journalistic integrity that was slowly established within the industry.  I believe that this was because of two factors: the limited amount of sources you could get your information from and the difficultly involved in switching sources.  While people could pick and choose what they read, heard, and watched, specializing in a specific slant was hard to do.  And even when you switched from watching CNN to watching ABCNews, there was not much difference, because news shows didn’t have to be.  There was no medium that was faster than TV.

Then the Internet came.  The Internet is great for all sorts of reasons, but one of the greatest advantages of the Internet is the ability to obtain information in the easiest means possible.  Today, the amount of information on the Internet, whether it is true or false, is almost limitless.  If you want to find information to validate any claim that you make, chances are you can find it.  And people like that! People like hearing what they want to hear.  I know I do. So people started to find only those sources that made them smarter.  From news sites, to blogs, to Facebook, and now to Twitter, people generally look at what interests them and abstain from what doesn’t.  In fact economists Jesse Shapiro and Matthew Gentzkow of the University of Chicago conjectured in their paper “What Drives Media Slant? Evidence From U.S. Daily Newspapers” that it wasn’t the newspapers that determined the slant of the readers, but rather the readers who drove the slant of newspapers.   From the NYTimes article about the paper:

As Dr. Shapiro put it in an interview, “The data suggest that newspapers are targeting their political slant to their customers’ demand and choosing the amount of slant that will maximize their sales.”

This choosiness though, has had a certain effect on news on the whole.  Because the Internet allows people to quickly pick very focused sources of information, other mediums had to come up with methods to compete.   Accordingly, TV and newspapers seem to be becoming more and more polarized.  An obvious example is Fox News vs. NBC News.  Both used to be closer towards the objectivity, but have realized that as readers become choosier – because they have more opportunities to be choosy on the Internet – the programs must also morph to fit the demand of the viewer.

I believe that more and more people are moving away from objective journalism, because objective news is no longer the substantive source of what people want to hear.   Before the Internet people did not have the fuel of niche blogs or programs to justify all their convictions, but rather, they only had their own intuition of how things should be that was then tested in newspapers, on TV and in the radio.  Sometimes your convictions aligned itself with the source and sometimes it didn’t, but this was the case for most sources.  Thus, people didn’t value a source on how much they agreed with it, but rather they valued the source on integrity, consistency and ability to report.  Now though, it is very different.  For the majority of people, the less that a source disagrees with your own beliefs; the more likely you are to revisit that source.  And, as evidenced by Dr. Shapiro and Dr. Gentzkow’s study, the sources realize this.

This is a bad direction for media in America.  We want Americans to have a multiplicity of views, not to be narrow-minded.  How is this possible though, when we only read the consenting perspectives?

Thanks goes out to Allan for edits.

Sorry for excessive metaphors related to trees, but it just seemed so fitting.

You see, for almost a year, Sachin (the other founder of Cog Tree) and I have spent every moment of our free time to the path of starting this company.  We felt quite nomadic during that time — we would leave our full-time jobs at the end of the day and meet at wi-fi cafes scattered across New York City.  We were heartbroken when our favorite cafe, Gramstand, closed.  I still miss getting a ginger mint honey iced tea and plowing through some work at 7pm.

For part of this time at the early stages of our company, Sachin lived in Brooklyn and I in Long Island.  Later, he moved to the financial district and I moved to Astoria, Queens.  Even when we settled in our apartments, we were always moving — always coming up with places to meet, always working remotely, always hopping on a subway or searching for a new wi-fi spot to work.

For the first few months of this, we brainstormed ideas and did research.  When we landed on the kernel of an idea we liked, we started hacking on a prototype.  And when we liked the results from that prototype, we applied to startup incubator programs.  Doing all of this while still maintaining a full-time job was demanding and definitely had adverse affects on our personal lives.  Our families complained of never receiving phone calls from us.  Our significant others got used to our distracted, “in the zone” looks as we focused on our computer screen.

April was a milestone month for us.  We finally had the balls to quit our full-time jobs.  We were ready to bootstrap this company with consulting money from part-time work.  But then, we were pleasantly surprised to learn we had gotten accepted by DreamIt Ventures in Philadelphia.  For three months, we would have the opportunity to work — full-time! in a real office! — on our company.  We would even be able to hire a third developer to join us on-site for the summer.  Truly a dream come true.

Last week, we finally moved down to Philadelphia, and Cog Tree moved into its shiny new office in the Philadelphia Science Center.  We’re now working side-by-side with nine other talented companies, and even living with the founders of two of them — Matt, Becki and Burak from Trendsta and Jack from Scribnia.  Our team also grew significantly, with one new full-time developer and two part-time ones.

Last week felt like the first week of real, sustained work on our project.  Rather than multi-tasking and squeezing in a few minutes here and there to work on our business, we were focused and determined.  We brainstormed ideas, surveyed users, hacked on code, and crafted an outline of our first deliverable product.  It felt great.  And it’s just going to get better as we move through the summer.

If you want to keep track of what we’re up to, definitely sign up for our mailing list at http://cogtree.com.

Welcome to the Cog Tree blog!

We’re excited to tell everyone about Cog Tree and the awesome projects we’re working on this summer.  Cog Tree evolved from a few different roots to tackle a problem that is affecting quite a few people and industries today.  And while we’re still just a young sapling, we’re working on growing into a robust Tree!

Whoever coined the “Information Age” knew what they were talking about.  Information is not hard to come by today.  From research to reputation management, gaining information about a topic is generally pretty easy.  Remembering my days of going to the university library to sit on the floor and flip through various academic journals reminds me of how access to information has changed.  Back then, you could search and not come up with a result for days.  Now, there are usually too many results for any given query.  While having a lot of information is better than having none, too much information can be overwhelming.  News sites, blogs, Google Alerts, RSS Feeds, TV, Radio, print magazines/newspapers — they all have more information about what you are looking for than you have the time in a day to read them all.

This is the exact problem that Cog Tree is tackling. We want to make sure that people can take advantage of the Too Much Information Age without worrying about the “too much” part.  Tracking clients, competitors, and sectors is time-consuming and expensive.  It can be easier though.  We’re building the tools to research, track and analyze the news.  At Cog Tree we want you to spend less time reading the news, and more time acting on it! In fact, that’s our motto!

Look for future posts from Cog Tree about ways we can make following media easier and more actionable than ever before.