“Shape” here doesn’t refer to just shapes, as in images. Rather, it also refers to structured arrangement. For instance, making heading text really large and making it stand out is part of “shape.” The CommunityWikiLogo in the top-right corner is an image, but it’s arrangement- that it’s in the top-right corner, and such a size, and related to the bar at the top, and the text down and to the left, seperated by headings- is all part of the “shape” of the page presentation.
You can construct visual language entirely without image. For example, you can just play with shape and text, and make very sophisticated visual language.
Visual Language is vast. It studies techniques found in schematic, architecture, movie technique, comics, visual arts, dance, image management (clothing), user interface, sign language, semiotics. Different people focus on different things, but the recurring message is that visual language is a language, and that it is continuously being developed in a cross-disciplinary way. It has grammar. It has vocabulary. There are aspects that are built in, and aspects that we just make up. There are dialects. And none of it exists in a vacuum: If someone makes something up in architecture land, it’ll probably appear in computer land later on. And if someone makes something up in the movies, the technique will probably appear later on in your UserInterface.
To get a feel for this, just look at Robert Horn's picture of the forces behind "Visual Language."
Note: Visual Language goes far beyond the replacement of words by icons!
To get a feel for how limiting that would be, take a look at the cute Zlango Icon Language:
The point is that it may be fun to look at, but it is actually less expressive than writing the text (unless you expect to be saving on mental translation costs if you’re not an English native speaker). Visual language is about being more expressive.
See Also: EmoticonsOnCommunityWiki
Visual Language also extends from informal napkin sketches to the mathematically-formal. In 1883, Charles Sanders Peirce (pronounced “purse”) invented the algebraic notation for predicate calculus as we know it today. In the late 1890’s he developed way of expressing what might be called “diagrammatical reasoning”, what he called Existential Graphs (EG). Essentially any statement in first-order logic could be expressed in either the algebraic or Existential Graph notation. EG could additionally express contexts by drawing an oval around an existing statement, essentially adding a metalevel assertion about the statement. For more information on EG, try Peirce's Existential Graphs - Readings and Links by Frithjof Dau. There’s a paper on the history of EG C. S. Peirce and the Quest for Gamma Graphs as well as a manuscript from Peirce himself, MS514: Existential Graphs, annotated by John Sowa.
A great deal of research has gone on in this area since the time of Peirce; the fields of mathematics, logic, and semiotics have been highly influenced by his work, often via other researchers who discovered his ideas.
One of those researchers, John Sowa, basing his own work on a combination of Peirce’s Existential Graphs and Artificial Intelligence’s semantic networks, developed a visual language called Conceptual Graphs, which he claims is “logically precise, humanly readable, and computationally tractable.” Sowa published “Conceptual Graphs” in 1984, though the book is now out of print. His 1999 book Knowledge Representation provides a very good introduction to Conceptual Graphs, as well as logic and ontology (both philosophical and computer-based). Conceptual Graphs can be converted to the Knowledge Interchange Format (KIF), which is a popular AI language that looks similar to LISP.
In CG, one can formally (i.e., mathematically-formally) express things like “Joe said, ‘I don’t believe in astrology, but they say that it works even if you don’t believe in it.’” (an actual example from Sowa’s book). We’re a very long way from having computers “understand” human speech (whatever “understand” might actually mean is up for grabs), but this is a way of constraining the way we express sentences (either textually or graphically) that machines can reason upon.
“And when there are images, the text that goes with the images should be integrated with the pictures. In five different tests, one group was exposed to text placed below the illustration, while the second group was exposed to text placed near the illustration. Although both groups saw identical text and graphics (with the only difference being placement of the text), in all five studies the second group performed better on subsequent tests.” – KathySierra 
Of special note is Visual:HornsVisualLanguageBook (“Visual Language: Global Communication for the 21st Century”). I think it is one of the most underlooked books in the field. And- judging by the importance of the field- perhaps in the world.
Far more visible is Visual:ScottMcCloud's worthy "Visual:UnderstandingComics." Perhaps that’s because Scott McCloud speaks the PlainTalk of everday comics readers, whereas Robert Horn’s used to talking to CEO’s and academics. This is not to dis either book; They are both excellent. Robert Horn’s just needs a little help in the visibility domain.
Oh yeah, and there’s a guy named Visual:EdwardTufte. He thinks about pictures and diagrams and bullet points. I don’t know- what he has to say is interesting, but I think that what Scott McCloud and Robert Horn has to say is much more interesting, and much more world-shaking. Somehow, everyone knows Edward Tufte, some know of Scott McCloud, but only a tiny few in the know have heard of Robert Horn. Chalk this up as more evidence that credit is not fairly distributed. (DeathToCredit?.)
Part of the reason Tufte is well known is that he’s published a number of truly beautiful and influential books on visual language, and has traveled and lectured widely. His books because almost standard reading among many graphic artists because they elucidated both the good and the bad of visual language in a very clear and compelling way. The most popular of his books, “The Visual Display of Quantitative Information”, is highly recommended for anyone who produces charts and graphs showing data.
Finally, your name is of special note. You can go to the Visual wiki, talk, and advance the field of Visual Language.
Really, it’s that simple.
These fields of thought are wide open, and they are, literally, (or perhaps I should say, graphically,) very beautiful.
Alas, the visual wiki is temporarily offline as of 2009-03-14. Sorry! I hope to have something online in a week. So … multiply by the Scotty factor and convert to the next higher unit of time … give me call if it isn’t online by the middle of May 2009. – DavidCary
A video that starts to talk about this a bit and has a good explanation of visual language. http://www.davegray.info/2008/04/11/the-whirl/
TODO more here…
We’ve flirted this for so long, I just felt we had to have the page.
When the technologies are in place to tear down the Visual:LongImageIncorporationProcess, the Internet is just going to totally light up.
We’ll have all kinds of mad diagrams in here; It’s going to be awesome. We will understand like we have never understood before. The tyranny of text will be over, and it will be amazing.
This is awesome. More related resources:
It seemed like back in 1999 that I received an invitation for this big conference in Japan that was being put on by Brian Eno and some other people, where they were going to explore the topic of VisualLanguage. I don’t know what all went down at that conference, but I am going to see if I can track down what it was all about, and if anythign came from it.