“Visual Language” generally refers to language that uses words, shape, and image. (That categorization comes from Visual:RobertHorn, and his definition of Visual:VisualLanguage.)

“Shape” here doesn’t refer to just shapes, as in images. Rather, it also refers to structured arrangement. For instance, making heading text really large and making it stand out is part of “shape.” The CommunityWikiLogo in the top-right corner is an image, but it’s arrangement- that it’s in the top-right corner, and such a size, and related to the bar at the top, and the text down and to the left, seperated by headings- is all part of the “shape” of the page presentation.

You can construct visual language entirely without image. For example, you can just play with shape and text, and make very sophisticated visual language.

Visual Language is vast. It studies techniques found in schematic, architecture, movie technique, comics, visual arts, dance, image management (clothing), user interface, sign language, semiotics. Different people focus on different things, but the recurring message is that visual language is a language, and that it is continuously being developed in a cross-disciplinary way. It has grammar. It has vocabulary. There are aspects that are built in, and aspects that we just make up. There are dialects. And none of it exists in a vacuum: If someone makes something up in architecture land, it’ll probably appear in computer land later on. And if someone makes something up in the movies, the technique will probably appear later on in your UserInterface.

To get a feel for this, just look at Robert Horn's picture of the forces behind "Visual Language."


Note: Visual Language goes far beyond the replacement of words by icons!

To get a feel for how limiting that would be, take a look at the cute Zlango Icon Language:

Fun to look at but not very expressive

The point is that it may be fun to look at, but it is actually less expressive than writing the text (unless you expect to be saving on mental translation costs if you’re not an English native speaker). Visual language is about being more expressive.

See Also: EmoticonsOnCommunityWiki

Mathematically Formal Visual Language

Visual Language also extends from informal napkin sketches to the mathematically-formal. In 1883, Charles Sanders Peirce (pronounced “purse”) invented the algebraic notation for predicate calculus as we know it today. In the late 1890’s he developed way of expressing what might be called “diagrammatical reasoning”, what he called Existential Graphs (EG). Essentially any statement in first-order logic could be expressed in either the algebraic or Existential Graph notation. EG could additionally express contexts by drawing an oval around an existing statement, essentially adding a metalevel assertion about the statement. For more information on EG, try Peirce's Existential Graphs - Readings and Links by Frithjof Dau. There’s a paper on the history of EG C. S. Peirce and the Quest for Gamma Graphs as well as a manuscript from Peirce himself, MS514: Existential Graphs, annotated by John Sowa.

A great deal of research has gone on in this area since the time of Peirce; the fields of mathematics, logic, and semiotics have been highly influenced by his work, often via other researchers who discovered his ideas.

One of those researchers, John Sowa, basing his own work on a combination of Peirce’s Existential Graphs and Artificial Intelligence’s semantic networks, developed a visual language called Conceptual Graphs, which he claims is “logically precise, humanly readable, and computationally tractable.” Sowa published “Conceptual Graphs” in 1984, though the book is now out of print. His 1999 book Knowledge Representation provides a very good introduction to Conceptual Graphs, as well as logic and ontology (both philosophical and computer-based). Conceptual Graphs can be converted to the Knowledge Interchange Format (KIF), which is a popular AI language that looks similar to LISP.

In CG, one can formally (i.e., mathematically-formally) express things like “Joe said, ‘I don’t believe in astrology, but they say that it works even if you don’t believe in it.’” (an actual example from Sowa’s book). We’re a very long way from having computers “understand” human speech (whatever “understand” might actually mean is up for grabs), but this is a way of constraining the way we express sentences (either textually or graphically) that machines can reason upon.

Learning And Visual Language

“Words + pictures > words alone” by Kathy Sierra “And when there are images, the text that goes with the images should be integrated with the pictures. In five different tests, one group was exposed to text placed below the illustration, while the second group was exposed to text placed near the illustration. Although both groups saw identical text and graphics (with the only difference being placement of the text), in all five studies the second group performed better on subsequent tests.” – KathySierra [1]

Also note the VisiBone products. They sell reference cards for current browser stuff: Color charts, character charts, Javascript summaries, HTML, CSS – everthing a web monkey needs. Look at how they compare two versions of the same product as an example for visual language in a hybrid environment. Given the constraints of typical browser windows, given that there are only two products to compare, they expertly juxtapose thumbnails, hinting at the richness of the product (use of colors, columns, fonts), with the text being short and to the point, leaving the rest up to the reader.

Names and Books of Special Note

Visual:RobertHorn is a major academic researcher of Visual Language. More recently, Visual:NeilCohn is an academic researcher who’s been writing a lot on the web.

Of special note is Visual:HornsVisualLanguageBook (“Visual Language: Global Communication for the 21st Century”). I think it is one of the most underlooked books in the field. And- judging by the importance of the field- perhaps in the world.

Far more visible is Visual:ScottMcCloud's worthy "Visual:UnderstandingComics." Perhaps that’s because Scott McCloud speaks the PlainTalk of everday comics readers, whereas Robert Horn’s used to talking to CEO’s and academics. This is not to dis either book; They are both excellent. Robert Horn’s just needs a little help in the visibility domain.

Oh yeah, and there’s a guy named Visual:EdwardTufte. He thinks about pictures and diagrams and bullet points. I don’t know- what he has to say is interesting, but I think that what Scott McCloud and Robert Horn has to say is much more interesting, and much more world-shaking. Somehow, everyone knows Edward Tufte, some know of Scott McCloud, but only a tiny few in the know have heard of Robert Horn. Chalk this up as more evidence that credit is not fairly distributed. (DeathToCredit?.) :)

Part of the reason Tufte is well known is that he’s published a number of truly beautiful and influential books on visual language, and has traveled and lectured widely. His books because almost standard reading among many graphic artists because they elucidated both the good and the bad of visual language in a very clear and compelling way. The most popular of his books, “The Visual Display of Quantitative Information”, is highly recommended for anyone who produces charts and graphs showing data.

Thinking Like Einstein: Returning To Our Visual Roots With The Emerging Revolution In Computer Information Visualization

Visual Language Wiki

Finally, your name is of special note. You can go to the Visual wiki, talk, and advance the field of Visual Language.

Really, it’s that simple.

These fields of thought are wide open, and they are, literally, (or perhaps I should say, graphically,) very beautiful.

Alas, the visual wiki is temporarily offline as of 2009-03-14. Sorry! I hope to have something online in a week. So … multiply by the Scotty factor and convert to the next higher unit of time … give me call if it isn’t online by the middle of May 2009. – DavidCary

Visual thinking as an approach to information organization and communication

A video that starts to talk about this a bit and has a good explanation of visual language.

TODO more here…

See Also

CategoryInformationVisualization VisualLanguageOnCommunityWiki


We’ve flirted this for so long, I just felt we had to have the page.

EvanProdromou indirectly prompted this in ContentAndCommunity.

We occasionally talk about VisualLanguage on this wiki; Alex- would you put “Visual” on the InterMap, with as the target?

When the technologies are in place to tear down the Visual:LongImageIncorporationProcess, the Internet is just going to totally light up.

We’ll have all kinds of mad diagrams in here; It’s going to be awesome. We will understand like we have never understood before. The tyranny of text will be over, and it will be amazing.

MurrayAltheim, thank you so much for the additional text on conceptual and existential graphs. I wish you were participating on the Visual wiki. :)

This is awesome. More related resources:

Gestalt Psychology

Thinking Like Einstein: Returning To Our Visual Roots With The Emerging Revolution In Computer Information Visualization

It seemed like back in 1999 that I received an invitation for this big conference in Japan that was being put on by Brian Eno and some other people, where they were going to explore the topic of VisualLanguage. I don’t know what all went down at that conference, but I am going to see if I can track down what it was all about, and if anythign came from it.

During a phone conversation with LionKimbro, I brought up MarshallClemens?. He has a great diagramatic process. I’ve added him to the “see also” section above.

I also added my friend CharlesCameron?’s HipBoneAnalysis? ideas.



Define external redirect: MarshallClemens DeathToCredit ThisPageIsDiigoAnnotated AssimilationContrastEffect CharlesCameron HipBoneAnalysis HipNone

EditNearLinks: UserInterface


The same page elsewhere: