Embedding Twitter discussions in a website with Yahoo Pipes #edtech #uoltech

Embedding Twitter discussions in a website with Yahoo Pipes #edtech #uoltech

So: you’ve been beavering away on Twitter, exchanging info, getting some good discussions going and generally being an active member of a community. Now you would like to bring that conversation to other people’s attention in order to get them engaged. If they’re on Twitter already, it’s not going to be difficult – just give them the hashtag (#uoltech in my case as I’m trying to grow the University of Leeds #edtech community) and they’ll be able to join in.

What can you do if those folks you need to reach are not on Twitter, however? What’s the easiest way they could stay up-to-date (if not participate)? In my case I already had a website to work with and a colleague suggested embedding the RSS feed from the Twitter conversation into the website. Brill idea, but what do you do when your RSS widget just tuts and nods in disapproval when you come with a search result from Twitter (because Twitter is a bit special and doesn’t actually do RSS feeds like everyone else – but I love it so much that I’m not going to hold it against it)?

Well, after tutting back at the RSS widget (Google’s, by the way), I went to Yahoo Pipes (another brill Yahoo creation which, because it’s so brilliant, I fear the gazillion-dollars/year Yahoo lady will try and sell off just like she’s doing with Delicious). I love Yahoo Pipes because they allow you to get content from pretty much anywhere (lots of places you wouldn’t normally think you could get content from, too, like static webpages), apply whatever filtering and transformational rules you wish and TaDa! your results display obediently and can also be viewed in several formats (RSS being one of them).

What I needed:

  1. the URL of the Twitter search for my hastag (#uoltech) in a format that Yahoo Pipes would like. This turned out to be http://search.twitter.com/search.atom?q=%23uoltech (thanks to everyone on the blogs for pointing out the ATOM search parameters)
  2. sign in quickly to Yahoo Pipes with my Yahoo credentials and build a super simple pipe that would take that feed I mentioned above (47seconds exactly)
  3. change the format of the output so that it starts with the author of the tweet thanks to the Regex code I found used by Rockfish Social in their Twitter RSS converter for Pipes – and that’s another cool thing about Pipes: there’s lots of code around to get you started, as well as good support and tutorials

What my super simple Yahoo pipe looks like

4. Run the pipe and get the RSS feed for it (click on the Get as RSS button in the new window – check out the screenshot below for alternative output formats for your Yahoo Pipe)

Running my Yahoo pipe gives this result. Nice!

5. Now the Google RSS widget on my website no longer complains. It does display the feed and if it could only get a move on and refresh more quickly, it would be brilliant (I know, I know, I should do some proper programming and write my own RSS widget…). But nevertheless, this is major progress!

So Thank you, Yahoo, for letting some very cool people on your payroll develop Pipes for the benefit of lay folks like myself and if you could find it in your heart to split with those very clever dudes and dudesses some of those ridiculous pay-cheques you’ve been throwing out of the window lately, the world would be a much better place.

EDIT: There’s no such thing as a perfect free lunch? While the Google widget in my webpage brings out the latest four tweets and clicking on any of them takes the user to a an individual Twitter page with that particular Tweet on and all the links working well, things get hairy when the user clicks on the title of my Google widget which takes visitors to the long list of results the Yahoo Pipe produces. I personally love this list because it gives a much more comprehensive image of what the community has tweeted. However, rather than sticking in a link to the actual tweet like the Google widget does, my Yahoo Pipe messes up the link by adding a reference to the tweet author’s image. So, instead of the link pointing to ?http://twitter.com/suchadrag/statuses/27011276257366016, it actually points to http://twitter.com/adobeconnect/statuses/27131728317382656,http://a0.twimg.com/profile_images/1152232005/adobe_connect_8_mnemonic_normal.png from Yahoo Pipes. Why? No idea but I am hoping I’ll find out soon and fix it because I guess not everyone will be willing, on the Page not found page they get by clicking on the link, to delete everything including and following the ‘,’ in the URL… Pipes is down at the moment so I can’t add any new filters to try and prevent this from happening… I think I’m going to go for a swim…

EDIT2: No swimming yet (story of my life)… I was intrigued by the ability to get the pipe output as a Badge and wondered what that was actually. It turns out it’s something very cool: Javascript code to embed the results into your website, so pretty soon it will be ByeBye Google widget. A little step in between for me is asking a cool friend to persuade the CMS he’s written and we’re using to allow me to add <script …></script> to individual pages.

What the Yahoo Pipes Badge looks like


EDIT 3: I hear that the Yahoo Pipes method is dead as a Dodo now, so please check out the short comment for a tip on what you could do now.