Quite often, we decide to organize stuff.

I generally leap right into organizing stuff into a hierarchy. And then I forget about the reasons why I needed to organize that stuff.

But there are alternative ways to organize things. Quite often they work better than a strict hierarchy. If I know the reasons for organizing something, then I can see which of these alternatives is better.

Too often data is forced into a strict hierarchy, even when the data doesn’t stay inside the Wiki:LimitsOfHierarchies. In those cases, HierarchyConsideredHarmful, since some alternative would be better.

Describing Shapes and Relations

A hierarchy’s shape is basically that it is a “tree” with a few things on top and a lot of things at the bottom. Arrows point down from the top towards the lower things. (Or vice versa; However you want to do it.)

A slightly weaker (less strict) form of hierarchy is the semilattice. This is like the tree, but where higher-ups can share access to lower-downs. (See TreeVsSemiLattice for some nice diagrams.) The semilattice has attracted attention here recently; ChristopherAlexander used the semilattice as a foil for thinking about cities in a City is not a Tree.

The most generic arrangement of things is the Graph. Any and every structure can be conceived of as a graph. A graph just means: “A bunch of things, and the relationships between them.” If you need a particular sort of arrows (for example, in your tree, you need arrows from higher-ups pointing to lower-downs,) you can choose between your undirected (no-arrows) and your directed (with arrows) graphs.

There are various other systems. You could be UsingTags. You can think in terms of tables, and call it “relational.” You can think about sets. Many of these arrangements are mappable to the other ones; They are just different expressions of the same basic things.



Intro by DavidCary, I wrote the section at the bottom, based on the collected notes on an older version of HierarchyConsideredHarmful.

So much of this stuff just begs for VisualLanguage. I’m having monitor difficulties, though, so I don’t have good access to my SVG tools atm.

Sticking to one hierarchy seems to be the real problem. UsingTags hides that we are using hierarchies anyway.

I think that we have to look for something like “incremental refining tagging schemes”.

We organize things to be able to find them again. While organizing we want to spend as little time as possible. So we tag with some keywords (ie portrait magnolia me).

Our AI tagging aid would expand this to portrait image magnolia tree thomas_kalka , using hierarchies (magnolia is a tree, portrait is an image).

This step could happen while tagging, by aggregation of tags from multiple users, or from user feedback while searching.

A capital idea!

It does seem that it should be possible to infer the tagging tree, especially given numerous people’s tags.

Hey- check this out! Here’s a newspaper article about it. It’s a way for infering connections between words via google.

Anothing thing that should be possible is using WordNet in all these tagging systems. If you enter “magnolia,” it could index it not just as magnolia, but the elements in it's tree as well.

Woe to the person who looks up “entity,” though…

In my opinion wikis would be the best fit to build this “multiple hierarchies”, taxonomies, or however we will call them. Poeple would choose their taxonomies by choosing wikis they like to participate. For things etc. wikipedia will possibly be the best fit. For using wikis as places to evolve taxonomies you need a concept to mark links with directions. Actually you can use the category system as a directed system. Wikis should support marking some of the links from one page to others as “directing to a broader concept” or “directing to a narrower concept”.

Users publish their prefered wikis. Using this information AI can then help to refine “tag” to “interwiki:pagename” in the first step. Using the directed links from the page, AI can expand the broader contexts or while showing search results ask to narrow contexts and assign these to the tagged pages.

We’ve usually thought about the LimitsOfHierarchy? here, but this makes good sense.

You could use a WikiFormat? or MachineCodeBlocks to store and collect the data from wiki. So, to mark directions, we could use a short WikiFormat?, and then spider wiki to build the database.

Our category system is not a directed system. Another problem is ChangeFailure; I know for a fact that there are a ton of errors in our categories. That said, I do believe that there is a TechnologySolution: We could normalize our form for describing categories on a page, and write tools (“wikilint?”) to help us normalize and establish consistancy amongst our categories. It would be a mix of manual and automatic labor. The tool would also be a good mid-way point towards writing towards writing the category spider, and towards category management in general. Then we could feed it the knowledge about our WikiFormat? for “directing to a broader concept” and “directing to a narrow concept.” (Hm, a general MicroFormat, for use across the web, could also work there.)

Did you see the WordNet? It already has a pretty detailed “things” hierarchy.

For that magnolia, it says:

  • angiospermous tree, flowering tree
  • tree
  • woody plant, ligneous plant
  • vascular plant, tracheophyte
  • plant, flora, plant life
  • organism, being
  • living thing, animate thing
  • object, physical object
  • physical entity
  • entity

I tend to disagree that tags are the same as hierarchy. Hierarchy means that each item is either contained in another item or does not have any common elements with the other one. In tags you can have tags for example ‘food’ and ‘plant’ and there are plants that are food, but there are also plants that are not food. You cannot have a hierarchy where you have tags ‘food’ and ‘plant’. You can have a DAG (think directories with symlinks) - but then you need to add new tags for all intersections.

It is still worth to note that faceted classification is not isomorphic with tagging. With facets you have your properties arranged into sets with tags you don’t.

BTW, there is a project underway already to add typed links to WikiPedia:

Plants may or may not be food, and food may or may not be plants, but that doesn’t mean you can’t use a hierarchy with tags.

WordNet will augment the tags “plant” and “food,” regardless.

What does it give you? If you do a search for “living thing,” you will get plants, people, bugs, etc., even if they didn’t tag themselves “living thing.”

Not totally useless, if you ask me, and most useful when you name specificly what you see: You type in magnolia, and shows up in a search for “tree.”

Just because “tags aren’t a hierarchy,” doesn’t mean we can use a hierarchy to aid our understanding of the tags.

I don’t think that graphs are reasonable general data structures. Although more general than trees, they have the problem of having two species of elements, typically “node” and “vertex” (mapped to “object” and “relationship”), which doesn’t really fit to modelling our thoughts or other complex structures. very often, when working with a map, the necessity comes up to treat a vertex as an object to add vertex-properties or vertex-relations and this is often not possible because it breaks basic assumptions about the system. I had a number of projects and problem during 20 years, where this experience repeated, again and again.

Then, a while ago, when I started to look into the OurCulture? community, I reread a google mailing-list “working in parallel” dealing with tools for thought modelling, containing 5000 messages from 5 years, and I was again pointed to the problem. But this time, from connecting the various viewpoint and the richness of assoziations, I was lead to a principle solution of the problem, a further generalization of the graph idea, which builds on a single element that can mutate to nodes or vertices as special cases and has flexible content (in a generalization of properties and relations). This basic element seems, when combined in fractal structures, to be able to model arbitrary thought structures. There is an intuitive graphical representation but yet no IT tool or project for implementation. It would be a kind of next generation for mind mapping.

It sounds great! I’d love to see it. :)

I think I understand the problem with the graph: It’s modelled under one set of organizing principles, and then, you have something that doesn’t fit, and you have a hard time then. (yes?) You have to restructure your whole graph, then.

Lion, I’d really like to have your opinion about it. The problem is that is is ready in my mind, notes and sketches - its really a simple concept - but it is not in a presentable or publishable form.

I think it would be nice to have at least a quick sketch of what we have so far. I made this image of basic data structures.

The sets is what you get with pure tagging, or categories. Every element can be a part of any set. This is the most general approach that you can get, but it means that you don’t have any additional information from it – just putting one tag doesn’t say anything about other tags. The sets don’t have to have the common part.

The tree is on the other end of the spectrum. It’s a rigid structure, just by placing an element in it you define everything. Thus, it’s small amount of work adding a new element, but the system is not very flexible, and you have to make decissions. Note that the direction of arrows is irrelevant – it can be deduced from the position of the root node. You can have several disjoint trees, then it’s called a forest.

The “tree of tags” approach gives you the third data structure, a DAG. This is how a sane UNIX filesystem looks like. Basically, you have a tree with some additional arcs. DAG stands for “directed acyclic graph”. There are no cycles, “circular dependencies”. It’s basically a model of an “IS A” type of relation. You can form this by first drawing a tree of tags, and then adding the nodes representing pages, and connecting them with the tags.

The last structure is a directed graph. There can be cycles and disjoint parts. Not sure what “meaning” could be attached to it, though.

Note, that all of these data structures can be modelled using any of the others. I can create a hierarchy of sets, by just following the rule that the sets must be contained in the bigger ones. I can simulate a set with a tree, for example by connecting the nodes to two special “in” and “out” nodes. All those data structures are equivalent in some way.

Now, what’s important is not the data structure alone, but the semantics behind it. What are the arcs, what are the nodes, what do the set elements represent, what does boing in given set mean.

I agree that the semantics are fundamental, but good visualisation is the goal to make it convincing or understandable, and data structure concepts are necessary for implementation. Without the last, one could just draw nice pictures and that’s it.

While I don’t want to nag at image of basic data structures which is quite beautiful, I remember an even more alaborate set of prototypical structures by AndriusKulikauskas (I’ll try to find it and provide the link) and I looking at it just felt “nice, but I’m not convinced that this is all”, I just don’t think that I’m able (or want to) present all my ideas through these models (or using MindMap or UML or …). This is a subjective matter, of course.

I think the fundamental actions “create object a” and “relate object a to object b” (either in a set or graph relationsship) are not conceptually rich, they are very near to “draw a box” and (“draw an arc from-to” or “draw a box inside”) but we do not think in these terms.

I think the fundamental semantic element is the process or algorithm, e. g. “draw boxes for all active members of cw” and “draw lines between members that wrote to same pages, thickness depending on the number of exchanges, the shorter the longer the relationsship”. This would produce a graph roughly depicting the social relationships of cw.

The thought “write a table (a vertical arrangement of boxes containing record data) from the lines, ordered by their intensity (algorithmic definition)” would be an example for a vertex->node transformation. Should be on a separate page ArcleModelling, I suppose.

Just a note – I updated the drawing to show an example of “set representation” that may be corresponding to the given example (of course different representations are possible, everything depends on the interpretation).

I found these on MinciusSodas?.

Yes, that’s the page I meant.

It seems to me that the data structures that we are talking about have two purposes, and that those purposes are conflicting somehow.

One of these is the presentation to human, that should reflect the way we think and perceive word, hopefully using some metaphors and parallels from the RealWorld, so that our instincts help us rather than get in the way. Things like spatian navigation, zooming interface, various forms of visualisation are created with this purpose in mind. There are also other representations tweaked for this, which are not visual (mathematical proofs written so that they are easier to understand, well-written computer programs), but I don’t know much about the techniques used.

The other purpose, however, is the manipulation of those sructures with “tools” that are “external” to our minds in some way – either formalized, like algorithms and mathematical proofs, or even hidden, like utility programs. Sometimes it’s hard to even store a data structure in a lossless manner, not to mention transforming it. I think that this is one of the main reasons why tree-like structures and hierarchies are so popular nowadays. There is a lrge array of tools we can apply to a tree, it can be easily searched and transformed. We can tell a lot of things about the structure itself.

It’s much, much harder to talk about algorithms in this way. Things get complicated, we no longer even know how to “traverse” it, not to mention the mathematical problems (algorithms hit the kind of infinity that math can hardly handle).

So it’s a constant struggle between what we mean, and what we can use. I think that wikis are indeed close to the “meaning” side, because their content is mostly interpreted by other people without the aid of much algorithmic tools. It’s visual, and organised only from the human point of view – for software it’s just a flat bunch of pages.

I would like to see how far one can go in this direction, toward “meaningful” and away from “useful” representations.

Define external redirect: MinciusSodas OurCulture WikiFormat LimitsOfHierarchy

EditNearLinks: MindMap RealWorld AndriusKulikauskas ChristopherAlexander