This is my own, subjective, personal account, of how web software development philosophy has gone:
Right now, this is basically where we're at. The web is the paragon of all life. All efforts at development, will begin life as an extension at the end of the network, and then, if it's any good, will survive the gatekeepers and the trials of time, and work it's way into the center of the web. The shining example of this, the white knight in glowing armor, the king of kings, is RSS. RSS is the model to follow.
There was a conference about a year ago; I can't remember the name of it right now, but I remember that SebPaquet mentioned it on his blog. At this conference, there was supposed to be a debate. It was a debate between the "edge of the network" guys, and the "integrationists." There were several dozens of speakers at the conference, if I recall right. There was only one problem: Nobody would speak for the integrationists side. That's how unpopular this view was: Nobody wanted to argue for the other side. I myself, at the time, agreed: "Of course now. There's nothing to be argued for in the other side. Gee, d'uh."
Now I think differently. I'm going to say: "Our history was wrong, and because of that, our reasoning was wrong."
While there are indeed benefits to NetworkedData? (a property of REST-HTTP/XML architecture, a property of HTML, not a property of HTTP), I'm going to say that we've lost sight of costs and limitations.
Let me propose an alternative view of history here.
Yes, I know that the development of technology is out-of-order. We're talking about the adoption points, not development points.
Now, what's interesting about this timeline?
It's about platforms.
Everything we see here, is a platform.
Let me give some points as example. Some things that we don't consider as platforms, but are platforms.
Note two things:
See: The web isn't magic pixie dust. You can develop clients that work outside of the web, and they work great. And extending "the web" from the outside really means creating an isolated platform on top of the web. Extending the network by protocol development doesn't really seem to be making much traction.
LocalNames is my personal effort. FOAF is a more popular one, but still utterly obscure, with no clear path for forwards motion. Single sign-on is still largely a dream.
Where do you see these sorts of things working? Within new platforms.
I am saying: The idea of extending the infrastructure of the web from the edges is a dream.
The platforms have been the real winners, as far as technology infrastructure developments have gone.
Take a moment to think about this, and to see if you can see the argument.
Think about it for a while, before proceeding on.
Now: What does this mean?
Let me give an example of how we can apply some of this thinking.
Your name is BayleShanks. You are very interested in online voting systems.
The traditional way to develop this would be to make a voting forum. Something that you could perhaps transclude into a web page. Or something where people would visit, to have their votes done. Write protocols (riding on top of other, successful, convenience protocols: such as HTTP, XML-RPC, or SOAP,) that allow you to connect the software with other software.
That is, a dis-integrated, protocol driven approach.
Now, we ask ourselves: "Okay, what does our new history & philosophy tell us?"
It tells us this: Make the voting system part of an integrated platform.
An integrated platform, like what? Like in Kuro5hin, or in these message board programs. In message boards, they're more like opinion polls. "What do you like on your pizza." But in Kuro5hin, they are built into the Kuro5hin platform itself. And it's wildly successful.
True, you can only use it on K5. But it provides the basis for further development. Ideas have been experimented and tried, and it works out pretty well. True, it's not as "nice" as having it immediately applicable everywhere, but it's pretty cool.
So, you're BayleShanks. What do you do?
What you do is you find some neat next-generation platform people are working on. Perhaps those guys working on Free Software 3D virtual worlds, or something. And you approach them, and you say: "Hey, guys. Guys. Do you mind if I integrate a voting and governance system into your project?" And they go, "Dude, wow. Someone thinks what we're writing is cool. He's got this crazy idea, but- let's bring him into the fold! It can't be totally sucky, right?" And they accept him. He implements his system in there, and there is now a generally useable (within this world, though) voting system. People can build atop it by building atop the platform.
Or, let's say they say "no." Then he forks the code base, and we here in CommunityWiki talk through his fork. When we have Ting meetings, we do it through the platform he made. Sort of like how we talk through this platform here called CommunityWiki (actually: Oddmuse).
See, somehow, we've had an illusion cast before us. That illusion is that: "If you're on the web, you're integrated." Because URLs (which are not specific to the WWW, I should point out) allow you to get to anywhere. Because most everything is on the web. Because AJAX is growing in strength, and soon to be augmented with SVG.
The illusion is broken, I think, though, if we notice the isolation of the platforms on the web.
Sure, web pages and web platforms can link to one another. If you make a link from CommunityWiki to CNN, that link will work. But, that's about the extent of the "integration."
It's like saying that the bash command line is "integrated" with all the programs on your computer, because you can start them from the command line.
That's connection, but not integration.
Now, I haven't fully sorted out the ramifications of this thinking, but these are the bases of it.
I would add something: I think that infrastructure protocols can develop after the platforms arise.
Local Names makes sense if you have lots of existing platforms that perscribe names.
If people find it useful to name blog posts, for instance, like people name wiki posts, and if short local naming for things becomes typical throughout different mediums, then the protocol for networking namespaces will naturally follow.
When you have a bunch of powerful platforms, then the protocols to connect them will follow.
Other example platforms to think about: UNIX. KDE. GNOME.
(Does anyone seriously believe they can significantly change the core bash experience for most users, by extending from the edges?)
Two other pieces:
Thus concludes my skewering of SideSystemsFirst.
Hmm … this seems to me like a black and white approach. Skype is only successful, since it can use the web for propagation. It is kind of a side-system, allowing voice interconnections to people known from the web, thus adding a new protocol. Otherwise it is a platform.
Look at delicious. It's a platform. But it is also a side-system, allowing to collect metadata for web pages, which are not writable. Instead of evangelising all web publishers to open their sites to public tagging it allows this by being a platform, implementing new protocols, apis etc.
I would say that SideSystems? are much likely a success, if they extend a already established platform. I mistilly remember, but there were some efforts to establish something like RSS and pings before (push channels). Rss seems to become successful in connection with the blogspehere and one special blogservice.
Alternatively to think about MachineCodeBlocks in wikis it maybe would be more efficient to try to establish an open platform (like delicious, with api and other interconnectivity) to extend any page with metadata.
I want to see more clearly what I mean by the following words:
I feel very strongly that I am observing something that is real. But my vision is blurry right now; I will need to spend more time on the thoughts, or read something clarifying, in order to see it sharply.
The problem is directly relevant to my work, and to the work of several other people who have been trying to add infrastructure to the web.
I think there's a distinction between building a platform on top of a platform (such as: building wiki on top of HTTP+HTML), and extending a platform (integrating SVG into web browsers).
Skype is not a side-system; It is a platform. That it's advertised and downloaded over the web is largely irrelevant, speaking technically. Technically, it's an entirely different system, and does not rely on the web.
The uber-platform is TCP/IP.
If I work against enough examples like this, I believe I will have a clear language that I can use to describe exactly what I mean.
Then I think I will be able to make my original point much more clearer, and the sooner we can stop trying to build FOAF + Local Names + all the myriad other infrastructure projects, and the sooner we can start collaborating on public platform projects.
MachineCodeBlocks builds a platform (MCB network) on top of an existing platform (wiki network). (That is, it works on top of any existing wiki.) But it does not extend the base platform. (When people write wiki, they do not say, "Ah, we must build in support for MachineCodeBlocks!") SVG extends the base web platform. RSS is built into all blogs and many community engines, and can now be considered as entering the base web platform.
Wiki is a platform that builds on the platform of the web. But it does not extend the web. (It does not extend basic web technology.)
Thanks for writing this, I've been thinking about these kinds of things as you know but they come into sharper focus with each discussion. Yes, we should definitely develop a PatternLanguage for this on wiki.
When people have a new feature that they want in wiki, they know that the political/transaction costs of getting their feature integrated into a current project are high, so they often just write their own WikiEngine. While it's true that eventually good feature ideas spread, it's annoying that each feature must be re-implemented on so many different platforms; this causes the spread of new wiki features to be shockingly slow.
Regarding the voting example (way to get my attention!):
I think it's true that if you want a feature idea to take off, it probably needs a "killer app", that is, a popular piece of software that uses that extension, thereby getting the word out.
However, I don't think this says anything about whether the implementation should be in the form of an isolated, integrated new platform, or an "edge of the network" extension.
If I am going to find project X and then help them implement a voting feature, there is nothing to stop me from using an "edge of the network" approach to give them their voting.
Another example: in this slashdot post slinked from the page SecondLife, you say with FOAF,
But, what is the difference between "mobilizing an army of people to make sure FOAF is integrated into a ton of stuff" and "mobilizing an army of people to make sure voting is integrated into a ton of stuff"? It's a lot of work in both cases.
Two more things to note:
Summary of the "killer apps" section:
Maybe it just takes a ridiculously long amount of time for new standards proposals to gain ground. So maybe FOAF hasn't failed yet, it's just slow. This might make sense since maybe a new proposal has to achieve a critical mass which is proportional to the number of programmers in the field. Since this number has exploded since the advent of the web, it would explain why it has been so hard to get extensions to the web to gain critical mass recently.
In an interview, Alan Kay said:
So, maybe the time it takes for new web ideas to become mainstream simply gets ridiculously long when there is a large culture of web programmers. That doesn't mean it won't happen.
Also, the same handicap might apply to new feature ideas implemented as standalone systems.
This topic is fundamentally related to many project that I'm interested in:
I agree that we've seen disappointingly slow progress in adding functionality through what you call the "edge of the network" approach. But the alternative is just too ugly. So I think it's worth it to continue to see if we can't make this dream work. But you're might, maybe it will never work and it is just a dream.
I'm still exploring this space.
(So, I think we should have this conversation.)
One argument that keeps appearing in my mind, is as follows:
That is: the programming environment will improve faster than the standardization efforts.
But is this true?
To evaluate that, we have to think about the costs and goals here. We should pay special attention to automated installation systems & component technologies, which could bring new life to the standardization approach, by making it easier to diffuse innovation.
You're right- we should probably define "integrate," because there are so many different types of integration.