We are now both technologically, and in some cases, socially, approaching some dimensions of goals and visions of OneBigSoup. Wiki engine scripts that are descended from WikiWikiWeb (or inspired by it) such as UseMod, OddMuse, ProWiki, are extremely efficient and flexible as wiki engines serving native wiki content. However, these CGI script-based wiki engines quickly struggle to deal with import and processing of external content via HTTP GET request.
The data ecology has changed since the inception of the WorldWideWeb. More and more, people now expect to be able to access and connect streams of data from multiple sources. The following is a proposed way to help wiki communities adapt to this change.
This page is an attempt to begin the discussion among the wiki community about:
My initial idea revolves around creating services using Wikipedia:Erlang programming language, and possibly the WebMachine? framework (http://bitbucket.org/justin/webmachine/wiki/Home ) to handle RSS/XML/JSON and other web service processing for wiki engines.
Users would still use wiki syntax (such as <rss> etc) to import content sets from outside of the wiki. The difference is that the CGI wiki script would now pass that request to a web service to process. The web service would then return either plain text, html, or some other format that the wiki engine can easily parse and display. The processing of RSS/XML and other data from external web services would happen in the WikiNetGrid node, which in turn could scale up to as many available nodes as needed to handle the request (Erlang can be configured to do this automatically).
AlexSchroeder suggests that I start in OddMuse by looking at using GetRaw?() in a module. I also this old thread http://www.oddmuse.org/cgi-bin/oddmuse/Comments_on_Transclusion
Next thing I need to learn is how OddMuse currently parses <rss> tag, and overriding this in my module. In the WikiNetGrid module, the “rss” tag will POST RSS url to a web service listed in the module, or config of wiki. The web service will get RSS data, parse, and return it as either HTML that OddMuse can include in page, or eventually possibly OddMuse markup.
Right now, I am thinking about how to extend OddMuse with a module or extension, to change the way it deals with these types of parsing and request (the module or extension would make the new way optional. It would override but not change the existing OddMuse code).
A wiki provides a changes-feed and various feeds from so called day-page-sets. These feeds are included via feedburner into a soup.io account which provides the soup-feed. The soup-feed is included via feedburner into twitterlive.net. Twitterlive triggers ping.fm. Ping.fm triggers the various accounts for the wikis on twitter, identi.ca, friendfeed, google buzz, jaiku, yahoo meme, yahoo profiles, etc. The ‘changes’ additionally go over a friendfeed group which adds an icon to the feed. Mybloglog imports the soup-feed but is not triggered by ping.fm. A web services included into a wiki page but not triggered by the soup-feed is amplify.
All inclusions use the <rss> method except the friends carrotmob-wiki and the poietic aggregator in the side-bar which are included via html.
Currently ping.fm posts corrupted html into the various web service. This issue will hopefully be solved. http://www.twitterlive.net/Help/why-do-ping-fm-posts-triggered-by-twitterlive-display-raw-html-46.aspx