WikiNetGrid

We are now both technologically, and in some cases, socially, approaching some dimensions of goals and visions of OneBigSoup. Wiki engine scripts that are descended from WikiWikiWeb (or inspired by it) such as UseMod, OddMuse, ProWiki, are extremely efficient and flexible as wiki engines serving native wiki content. However, these CGI script-based wiki engines quickly struggle to deal with import and processing of external content via HTTP GET request.

The data ecology has changed since the inception of the WorldWideWeb. More and more, people now expect to be able to access and connect streams of data from multiple sources. The following is a proposed way to help wiki communities adapt to this change.

This page is an attempt to begin the discussion among the wiki community about:

  1. Hybridizing wiki engines, so that certain actions are handed off to an outside service for processing, and needed text is returned back to the wiki-engine for display
  2. Pooling surplus resources among interested members of the community to create a scalable hardware grid that provides the service mentioned above, and is available to sharing participants.

My initial idea revolves around creating services using Wikipedia:Erlang programming language, and possibly the WebMachine? framework (http://bitbucket.org/justin/webmachine/wiki/Home ) to handle RSS/XML/JSON and other web service processing for wiki engines.

Users would still use wiki syntax (such as <rss> etc) to import content sets from outside of the wiki. The difference is that the CGI wiki script would now pass that request to a web service to process. The web service would then return either plain text, html, or some other format that the wiki engine can easily parse and display. The processing of RSS/XML and other data from external web services would happen in the WikiNetGrid node, which in turn could scale up to as many available nodes as needed to handle the request (Erlang can be configured to do this automatically).

Notes

Development status

AlexSchroeder suggests that I start in OddMuse by looking at using GetRaw?() in a module. I also this old thread http://www.oddmuse.org/cgi-bin/oddmuse/Comments_on_Transclusion

Next thing I need to learn is how OddMuse currently parses <rss> tag, and overriding this in my module. In the WikiNetGrid module, the “rss” tag will POST RSS url to a web service listed in the module, or config of wiki. The web service will get RSS data, parse, and return it as either HTML that OddMuse can include in page, or eventually possibly OddMuse markup.

Right now, I am thinking about how to extend OddMuse with a module or extension, to change the way it deals with these types of parsing and request (the module or extension would make the new way optional. It would override but not change the existing OddMuse code).

Thanks Sam for this page. It applies to the kabo-wiki-hive (amongst others). This is a short introduction on how wikis in this hive interact with web services. The carrotmob-wiki is an example.

A wiki provides a changes-feed and various feeds from so called day-page-sets. These feeds are included via feedburner into a soup.io account which provides the soup-feed. The soup-feed is included via feedburner into twitterlive.net. Twitterlive triggers ping.fm. Ping.fm triggers the various accounts for the wikis on twitter, identi.ca, friendfeed, google buzz, jaiku, yahoo meme, yahoo profiles, etc. The ‘changes’ additionally go over a friendfeed group which adds an icon to the feed. Mybloglog imports the soup-feed but is not triggered by ping.fm. A web services included into a wiki page but not triggered by the soup-feed is amplify.

All inclusions use the <rss> method except the friends carrotmob-wiki and the poietic aggregator in the side-bar which are included via html.

Currently ping.fm posts corrupted html into the various web service. This issue will hopefully be solved. http://www.twitterlive.net/Help/why-do-ping-fm-posts-triggered-by-twitterlive-display-raw-html-46.aspx

(CommunityWikiFooter)

Define external redirect: GetRaw WikiCopy WebMachine

EditNearLinks: UseMod OddMuse WorldWideWeb ProWiki WikiWikiWeb

Languages: