FrontPage SiteMap RecentChanges HowTo Blog

Matching Pages:

RSS

WikiGateway

WikiGateway is a library of functions which allows you to act as a client to a wiki website, executing operations like getPage, putPage, and getRecentChanges. WikiGateway translates your requests into the idiosyncratic HTML form interfaces of various popular wiki servers. Basically, it does Wiki I/O and import/export.

All text on this page is PrimarilyPublicDomain.

InterWikiSoftware:WikiGateway

It can be used either as a Perl module or as a web service providing XmlRpcToWiki.

Currently it only works with UseMod and MoinMoin, but contributions to make it work with other wikis are welcome.

Potential applications

See InterWikiSoftware:WikiGatewayMotivation for motivation, and InterWikiSoftware:WikiGatewayGeneralDiscussion for discussion, including security implications.

Present Work

News

OK I finally made some Debian packages. All of WikiGateway isn’t packaged yet, but the most important parts are. Namely, the commandline client, the Python module, and the Perl module. If someone is running Debian, could you please test this for me to make sure I got it right? Just follow the instructions below to install, and then do the following to test:

/usr/bin/wikiclient --type=usemod1 read http://www.usemod.com/cgi-bin/mb.pl:SandBox

if everything is working, that should dump the content of MeatBall:SandBox to STDOUT.

How to install the WikiGateway .deb packages (assuming you are running Debian)

(1) Make sure you are using Debian’s Unstable distribution; some of WikiGateway’s dependencies aren’t in Stable yet (as of Mar ‘06).

(2) Add the following line to your /etc/apt/sources.list:

deb http://interwiki.sourceforge.net/debian/ ./

(3) Do this:

apt-get update
apt-get install python-wikigateway libwiki-gateway-perl wikiclient

(or, if you only want the command-line client or only the Python module or the Perl module, just pick one of those packages; there are some dependencies between them, but apt-get will figure that out for you and install what’s needed)

Email Lists

For general discussion of InterWiki stuff:

For users of WikiGateway:

Testing

LionKimbro is helping BayleShanks test his system, by running it himself, and trying it out from different systems.

Please join us!

Atom

WikiGateway partially supports Atom. That is, it allows a third party to set up a gateway server which effectively allows users to read and modify a wiki via the Atom protocol. All without touching the code of the WikiEngine or even having access to its server.

See InterWikiSoftware:AtomGateway for details. See AtomWiki:AtomWikiGateway for a demo.

Command-line wiki client

There is a command-line frontend to WikiGateway in progress. Actually, it seems to work, but unit tests and documentation haven’t been written yet.

Synopsis:

			wikiclient --type=usemod1 read MeatBall:SandBox
                        wikiclient --type=usemod1 read http://www.usemod.com/cgi-bin/mb.pl:SandBox
			wikiclient --type=usemod1 write http://interwiki.sourceforge.net/cgi-bin/wiki.pl:SandBox	     
			wikiclient --type=usemod1 rc MeatBall
			wikiclient --type=usemod1 allpages http://interwiki.sourceforge.net/cgi-bin/wiki.pl

Information to be read/written is read from STDIN/printed to STDOUT. Note that InterMap monikers can be used as shortcuts for URLs. An optional UseMod style InterMap file is needed for this.

Raw Notes

I think you need “apt-get install libxml-rss-perl” on Debian.

Thanks; yes, you do; the above discussion is somewhat old (I’m about to delete it). Currently, the easiest way to install the core Perl WikiGateway library is to either use the libwiki-gateway-perl unofficial Debian package (get it from here), or to install the Wiki::Gateway module from CPAN. Either of these methods auto-installs the necessary dependencies. I haven’t yet gotten around to packaging the Python API, the command-line client, the XML-RPC API, the WebDAV API, or the spamclean.py script.


I like the idea of the BannedContentBot, which uses this. But if I understand correctly, the gateway functions could easily be used to build wiki sp@mming bots too. Not sure what you can do to avoid this, apart from not releasing it publicly. Maybe you should bundle it with some trojans/spyware which awakens when someone tries to make a page edit containing banned content :-) – [Halz]

That’s correct, it could definitely be used to automate spam or other malevolent attacks. See InterWikiSoftware:WikiGatewayGeneralDiscussion for the discussion about this (i.e. I’d like to redirect this discussion over there, in the spirit of ConsolidateInformation).


Questions

BTW I’ve promised to you on WikiSym to “gateway” the ProWiki software. What would the best way to go about it? Does the official UseMod release include an implementation? Or is it in the sourceforge repository?

Great, thanks! Hmm lemme see. There’s a couple ways we could do this. I guess the first questions is, does ProWiki already have an API of any sort? And the second questions are, what OS do you develop under, and do you know Python?

The reason I’m asking about the OS is because WikiGateway has a bunch of dependencies. It seems to me that the easiest way to deal with this is by using a GNU/Linux packaging system to make an easy-to-install package containing those dependencies. But if you use Windows that won’t work. Lion’s told me about an up-and-coming Python packaging system, so maybe I should learn to use that instead.

The reason I’m asking about the language is that the drivers for specific WikiEngines are written in Python, but that may be too much work for non-Python users. I’m considering adding another layer of abstraction (well, actually, just a special, configurable driver), and then just asking you to give me URL templates and regular expressions that will be used to get/post pages, parse recent changes, etc.

Hmm… in fact, right now I just realized that if I can get it down to using templates and regexps, maybe I could make a little CGI script that runs on my server that lets you test out the regexps. This way you can debug your driver without actually installing WikiGateway or editing any code.

Maybe I should also write a few pages of documentation for wiki engine developers who want to create a WikiGateway driver for their engine.

So, in any case, I’ll be doing some work to make it easier for you to make a driver for ProWiki.

My servers run Linux, ProWiki is Perl, so I will use Perl to implement the ProWiki / WikiGateway driver. Have you by-passed the UseMod engine and directly accessed the file base?

I thought the gateway API would be wiki command line driven, so that the drivers are built into the wiki engine and accessed by wiki commands like wiki.cgi?action=gateway&cmd=index&param=xyz . Is there no command line interface spec?

If there isn’t this is a deficiency, because the wiki engine regulates the access to wiki pages in many useful ways.

The WikiGateway “drivers” are client-side adaptors. The reason you would write a driver is if you didn’t want to change ProWiki at all but you still wanted clients to be able to communicate with it. For instance, the UseMod “driver” runs on the client side and pretends it is a wiki user; i.e. it requests pages from the wiki and submits changed pages by filling in the normal HTML edit form.

If you’re willing to change ProWiki, you don’t need to worry about writing a client-side driver (unless you wanted to make it easy for clients to talk to old ProWiki servers that don’t upgrade). I think changing ProWiki is just as easy, and better, than writing a driver – sorry, when i was talking about writing a driver, i was assuming you didn’t want to change the server. I agree that it’s good for the wiki engine to continue to regulate access – for instance, if a client submitted a page using the “wiki command line”, the server should check that page however it normally does for humans – for instance, if the wiki has a spam blacklist and the page contains spam, the wiki should deny the request to change the page.

So which spec to use? There are a few competing standards for that sort of thing – WikiGateway can speak many of them – and most WikiEngines don’t speak any of them (which is the motivation for WikiGateway – if all the servers implemented a standard spec, the clients could just use that protocol, and not bother going through WikiGateway).

If you want to implement a “wiki command line”, that’s great and I encourage it; that’ll make it easy for WikiGateway to use ProWiki, since WikiGateway can already speak most of the standard wiki interface protocols.

The standards I am aware of include WebDAV, Atom, and WikiRPCInterface2. WebDAV and Atom aren’t designed specifically for wikis, which has the benefit of making them more useful, but the dis-benefit that someday someone will have to come up with a standard mapping of the WebDAV and Atom “data model” onto the naive wiki “data model” (for example, how do you fetch “recent changes” over WebDAV? If you do it by requesting properties, what is the name of the property that corresponds to the “change summary”?)

There are efforts to create new standards too, such as WSR 3 and MeatBall:WikiMachineInterfaceStandard.

I think the simplest one for you to implement right now is WikiRPCInterface2. I think the “best” one right now, but which may take a little more work for you (ask AlexSchroeder how much, since he’s worked on this), is WebDAV. Hopefully in a year or so we’ll have a RESTful wiki standard, probably coming out of WSR 3.

I’m working on the ZergCreep; I’m solving the problem of: “If a change happens, but the thing that’s supposed to happen when the change happens fails, you need to try again, up to some timeout date.” It involves a little reworking of the code base.

I’m putting off WikiGateway integration, until I get the basic functionality down.

Discussion moved to WikiWindow

Regarding WebDAV: I gave up on fixing my implementation because the two most useful agents – OSX Finder and Windows Explorer – are not standard compliant in a useful way, as far as I can tell. My implementation worked well with the command-line client cadaver. There more about all of this on my web pages.

See Oddmuse and WebDAV – lots of notes and pointers to more info.

Regarding WebDAV; yeah, the WebDAV standard doesn’t seem to have taken off, does it? I used to be a cheerleader for it just because it was standard, and I still am I guess, but someday I’m going to take a look at it and at its deficiencies and decide if I think that it is too flawed to work (maybe its flaws are what prevents many projects from using it). Dunno what its flaws are but I’ve heard that Sam Ruby thinks its too heavyweight and I know that when I have experimented with it with WikiGateway, it seemed rather slow (that may have been the fault of the implementation that I was using though).

I have heard about various projects using WebDAV, but I also keep hearing about partially compatible implementations, suggesting that maybe the actual standard has something wrong with it that people don’t want to follow.

And I don’t hear much about DeltaV being used, which is too bad because that was the part that could really help wikis.

But maybe I’m being pessimistic. Subversion and Jarkarta Slide use DeltaV.


CategorySyndication CategoryInterCommunity CategoryWikiTechnology

EditNearLinks: XmlRpcToWiki WikiEngines ProWiki UseMod PrimarilyPublicDomain CategoryInterCommunity MoinMoin WikiServer WikiEngine

Languages:

The same page on other sites:
MeatBall:WikiGateway