SchematicMedium

“Schematic Medium” is a term coined by me, LionKimbro.

It’s an answer to the problem of VisualProgrammingLanguages? – specifically: "If visual programming languages are such a great idea, then where are they?"

The speculation is that there’s nothing wrong with visual programming languages themselves (as previous analyses have focused,) but rather, on the lack of a generally applicable schematic programming medium.

The proposal is to develop a schematic medium, fill the gap, and then promote the development of tools and languages that are expressed within this medium.

This is a very specific idea, so I need to patiently, and step by step, explain it.

"What is a Medium?"

To pump the intuition, I’ll start out and say: Text files are a medium.

Now if you get x5 text files and put them before me, and say, “Is this a medium?”, I’d say “No, that’s x5 text files.” But text files, considered abstractly, are a medium.

OK, now I’ll go from the concrete example (text files,) to the abstract (a “medium”):

A medium, by my definition here, has the following attributes:

Here’s how “Text Files” stacks up:

The Ecology

Another essential dimension of what I’m calling a medium, is that it has an ecology.

The “ecology” of a medium includes:

Let’s go concrete again, and look at the ecology of Text Files:

Question: off the bat – how many text editors do you think that there are, in existence, that can work on pretty much any text file?

Thousands? Tens of thousands? That might be the ballpark.

Question: off the bat – how many text manipulation tools and libraries do you think that there are, in existence, that can work on pretty much any text file?

Tens of thousands? Hundreds of thousands? That might be the ballpark.

The point is that text files are a medium, and that there is a rich ecology of tools, around the medium of text files.

Here’s a little diagram, encapsulating this idea graphically:

SchematicMediumImage001

The idea is: There’s a simple conceptual “medium” in the center, and it’s surrounded by an ecology of editors and tools.

The Speculation

Now remember, I’m introducing this cut called a “medium,” but I’m doing so in service of a very specific question: "If visual programming languages are so great, where are they?"

There have been a great many analyses of why, after some 60 years of programming, text files are still all the rage, and visual programming languages are still a relatively niche solution.

My belief is that we’ve mistargeted the solution. We kept thinking that it was about a visual programming language, whereas I believe that the problem is really the lack of a visual programming medium.

And by focusing all time and effort on the language, we missed the actual problem, which is that there is no visual programming medium.

An Inventory of Visual Mediums: Bitmaps

Think about it.

There are visual mediums.

Let’s look at our qualifications again:

(Example P1 PBM example, from Wikipedia:)

P1
# This is an example bitmap of the letter "J"
6 10
0 0 0 0 1 0
0 0 0 0 1 0
0 0 0 0 1 0
0 0 0 0 1 0
0 0 0 0 1 0
0 0 0 0 1 0
1 0 0 0 1 0
0 1 1 1 0 0
0 0 0 0 0 0
0 0 0 0 0 0

Let’s look also at the bitmaps ecology – is there such a thing? YES, there is. There’s an enormous number of tools out there for converting one bitmap file to another, to darkening or shading or hatching or outlining or – any kind of operation you want to perform on a bitmap, you name it, there’s a tool for it. Tools for use from the command line, tools with their own environments, and tools for programmatic manipulation. It’s all there.

And there are editors galore. There are heavyweights like PhotoShop, the GIMP, and myriad other tools, there are also lesser known tools – pixel editors, bitmap font editors, all kinds of things.

There is a very rich ecology for working with the medium of bitmaps.

An Inventory of Visual Mediums: Vector Illustration

There is another broad visual medium, and that is vector illustration.

I’m getting bored, so I’ll leave it as an exercise for the reader to identify why Vector Illustration is a medium.

But go through the checklist:

And then go through the ecology checklist:

If you go through this, you find the evidence that vector illustration is a medium.

It’s a weaker medium – it’s more complex, and there are fewer tools, not as many conventions, … But we can see a medium there.

Another way to see that it’s a weaker medium – You can copy and paste text between different text editors through the clipboard, and you can copy and paste bitmap graphics between different graphics programs through the clipboard, but – vector illustration elements? How would those even be expressed in the intermediary? I think the Microsoft ecology might allow some of that internally, … (strangely, it seems that “tables” might be a medium, too, …) …but not so much cross vendor.

So I’m going to judge that as things stand, vector illustration is a weaker medium.

But Where's the Visual Programming Medium?

OK, this is all nice, but what’s the medium for visual programming languages?

Every time we see a visual programming language, it’s expressed in an editor that is tightly confined to that specific programming language.

The vast majority of programming languages are expressed in TextFiles?.

SchematicMediumImage002

This solves a very large number of problems:

On the other hand, consider the visual programming languages:

SchematicMediumImage003

The designer is immediately hamstrung – the designer can’t even express programs in the designer’s language, from the get-go.

And consider too, all the additional tasks the designer has on his or her hands:

SchematicMediumImage004

So my proposal is that the real reason we don’t have visual programming languages, isn’t because visual programming languages are intrinsically a bad idea.

It’s not because they can’t or don’t work.

Rather, the reason we don’t have a plurality of great visual programming languages, is because we simply haven’t developed and figured out a schematic medium.

Qualities of a Schematic Medium

new: 2021-02-02 13:25 UTCLionKimbro:

I’m getting tired, so I’m going to have to flesh out the rest of this later.

But I’m going to rewrite what I wrote before, about what I believe are the essential elements of a Schematic Medium, and why I believe that they are so essential.

  • text – text must absolutely be an essential part of the medium
  • lines, ovals, rectangles – these elements must be part of the medium
  • icons – I believe that custom created icons, built into the editor and expression of the program, should be a fundamental part of a schematic editor.
  • grid alignment – just like in text files, everything MUST be grid-aligned (unlike a vector illustrator)
  • grouping – the capacity to make logical groupings should be part of the system

Also, some of the conventions that should abound around editors:

  • text search – the capacity to search the document for text
  • multi-buffer / multi-file support – like in EMACS or Notepad++, and unlike in Notepad

I’m also going to argue why it is that vector illustration is not a schematic medium, although it might look like one presently.

I’m looking for something that is specifically made for programming. A more technical term might be, “A programmer’s schematic medium.”

Discussion

new: 2021-02-02 13:25 UTCLionKimbro:

I have a basic request, – before going into this – that comments not be placed in-line to a bullet list.

We used to have a page that described this, I can’t remember where, – but my request is that points be added to a new discussion thread item, rather than in-lined everywhere.

I’m going to re-locate the elements of TimurIsmagilov’s response, here, now:

  • Q: “What is a foil” ..?
  • Q: On the subject of genericity: “How many kinds shall be supported?”
  • note on re-representation: “For example, a line connecting a class and a superclass. A schematic editor could highlight such line with color, curve or whatever. It’s like syntax highlighting in text editors.” – ‘’I think that you added this part? I can’t get at the page history for some reason, so I can’t tell.’’
  • Q: On the subject of text: “What language?”
  • note on lines, ovals, rectangles – “See re-representable point. I see a contradiction.”
  • note on grouping: “A ZoomableUserInterface would fit nicely.”

To answer these questions:

A “foil” is something that is opposed to something else, and in opposing the something else, serves to more clearly define it. It’s a word usually used to refer to characters in a story – One character is the opposite of another character, and may serve the purpose of allowing that character’s characteristics to show up.

With text files, they are simply parsed. Now, there’s a wrinkle in this – there are two (at least two) newline formats in use in text files: NL, and CR+LF. So that shows a degree of unwanted complexity. But overall, there’s a simplicity to newlines. “The exception that proves the rule.”

As for your questions and comments on genericity and re-representation, I understand neither your questions nor the contradictions you perceive.

I want to be really clear about this, it’s a super-important point of the entire system:

  • The system must support text.
  • It must support lines.
  • It should support circles, ovals.
  • It should support squares, rectangles.

This is kind of a “minimum” of what a schematic even is, really.

Remember, the goal is to be able to be a medium that many, many programming languages can be expressed in.

  • Medium: Text
    • Perl
    • Python
    • C++
    • C
    • C#
    • Lisp
  • Medium: Schematic
    • language-that-doesn’t-exist-yet #1
    • language-that-doesn’t-exist-yet #2
    • language-that-doesn’t-exist-yet #3
    • language-that-doesn’t-exist-yet #4
    • language-that-doesn’t-exist-yet #5

Part of the reason that text files work so well as a medium for programming languages, is that it is highly versatile using standard recognizable pieces: letters, spaces, arranged in a grid.

A schematic language needs to be similar – and perhaps also needs to be grid aligned. The focus is on extremely simple pieces, arranged in a grid.

So:

  • lines
  • rectangles
  • ovals

Simple. Simple.

No semantic meaning.

Text does not have semantic meaning.

The semantic meanings are for the language to specify, not for the medium to specify.

It must be similar with a semantic medium.

I may be over-stating it, but if I do, it’s because this is the most important distinction.

new: 2020-12-25 07:46 UTCTimurIsmagilov: To a future reader, that discussion is placed at TurnBasedVsInterruptedThreadMode. To a future reader, reading this discussion would be not so handy because I’m asked to abandon my tree discussions :-(

I now understand what is foil, thanks.

As for the rest, how does simplicity and presence of different 2D shapes fit together? Rectangles and ovals are not simple nor semanticless. A language A may use ovals for function declarations or something, and language B may use ovals for function invocations. Both uses have semantics but the semantics are totally different. If this is ok, then it’s like writing text like that in the text medium (up-to-down lines going left-to-right):

i l w n x e t
t i r g t  
  k i   l t
i e t t i h
s   i e k a

A bare minimum would be, in my opinion, this:

  • Text block
  • Lines (connecting different blocks)
  • Shape block (encasing a mesh or something) All of the elements can have types. For example, this shape is a function declaration, this text is an id, this text is a string, this line connects function declaration with its body.

Ovals, squares, rectangles and curves should be out of medium. As I said, it is like syntax highlighting in text medium.

new: 2021-02-02 13:25 UTCLionKimbro:

Thank you for, for at least in this conversation, agreeing to use a TurnBasedThreadMode. We’ll have the other conversation over in TurnBasedVsInterruptedThreadMode.

Now I’m trying to understand your question --

“As for the rest, how does simplicity and presence of different 2D shapes fit togther?”

I don’t quite understand your question. Are you saying, “How are 2D shapes simple?”

Here’s what I mean by 2-D shapes:

  • LINE, x0, y0, x1, y1 – the fact of drawing a line, a column and row for one end of the line, and a column and a row for another end of the line
  • OVAL, x0, y0, x1, y1 – (same, but for an oval)
  • RECT, x0, y0, x1, y1 – (same, but for a rectangle)

I think that these are simple shapes.

If y0, x0, x1, y1, are all integers, then there is also a simplicity in that they will correspond to one another (they will not be off from one another by, say, 0.034 places,) and they will correspond with proximately placed text.

It’s simple because:

  • it’s clearly defined
  • it has no specific color
  • it aligns with other entities in the drawing space
  • there are only three variety of things
  • it fits well with almost all existing graphics libraries

Let me contrast them with more complex things:

  • a graphic image, with 8.8.8.8 RGBA color, 243 pixels wide, 132 pixels tall, positioned arbitrarily on the drawing canvas
  • a bezier curve, with a gradient or a pattern used to describe its colorations

These things are complex because:

  • they arouse a lot of questions in their implementation, that are often not trivially deducible from an examination of their in-code representation – for example, with the graphic image, we have room to ask about: what’s the byte-packing scheme? what about palettized color? what about aligning memory, or padding cells for rapid indexing? what do you do on monitors that can’t display all those colors? how do you make use of the alpha bits? what about different methods of super-imposing images on one another?
  • specifying colors, now editing environments and such cannot color the element without changing its meaning (whatever the color originally stood for, in the mind of the author of the color on the element)
  • alignment now must be explicitly declared, because it will not be trivially obtained; there will be no standards or conventions of alignment, unless explicitly forced on the user by tools or what not; alignments cannot be relied upon in the general case
  • I can imagine that there would be a huge proliferation of drawing elements, that all demand implementation in all tools that claim to support the medium
  • tools will be “correct” or “incorrect,” based on how accurately or inaccurately they can represent the detailed particularities of the complex drawing elements

These concerns evaporate, in the main, by using a very simple format for depicting lines, ovals, rectangles, and aligning them to a grid, just like text is grid-aligned in a text file.

new: 2021-02-02 13:25 UTCLionKimbro:

“Rectangles and ovals are not simple nor semanticless.”

No – they are. By the way I am using these words, they are – if you disagree, it’s because you don’t understand my point, not because of anything about these words.

The letter “A” is not semantic. It’s a character, but not like the character 木 in Chinese or Japanese, which specifically means “tree.” The letter “A” has a phonetic suggestion, in most languages, but those phonetic suggestions vary by language, and besides, a sound, except in the specific case of onomatopoeia, does not carry a semantic meaning.

Similarly, rectangles and ovals, I declare, are not semantic.

If you draw a house, using lines and rectangles, then now it has a semantic meaning. If you define a language, then it can have a specific semantic meaning, particular to that language.

Do you understand what I’m saying? It’s okay for you to disagree, and we could have a conversation to explore or overcome that point of disagreement, but before you disagree, I need to be clear that you understand what I am saying here, first.

new: 2021-02-02 13:25 UTCLionKimbro:

“All of the elements can have types. For example, this shape is a function declaration, this text is an id, this text is a string, this line connects function declaration with its body.”

OK, here – remember, I’m talking about the difference between a “medium,” and a language.

By far, most programming in the world is expressed in text files.

The text files, very importantly, have no concept of “types.”

When I write a Python programming, and I write:

x = 13
y = "bar"

…the text file itself, has no concept of types. It only sees characters, which are encoded by an encoding system – typically either 7-bit ASCII or UTF-8 Unicode, these days.

But the text file itself doesn’t have an encoding concept of an integer type, of a float type, of a string type, or of any other kind of type.

Where then, does the concept of “type” appear in the Python programming language? Well, it’s precisely that – it’s in the concept of the “language,” not in the concept of the “medium.” Does this make it any clearer?

new: 2020-12-25 07:46 UTCTimurIsmagilov:

Are you saying, “How are 2D shapes simple?”

Yes.

After reading your message, I finally understood your idea. You just want to place shapes on text-file-like grid. Well, simple indeed, not gonna lie. Defining every shape with two coordinates is a nice idea. You can also add a trapezoid and parallelogram there, by the way.

No – they are. By the way I am using these words, they are – if you disagree, it’s because you don’t understand my point, not because of anything about these words. <…> Similarly, rectangles and ovals, I declare, are not semantic.

I now understand your point.

Do you understand what I’m saying? It’s okay for you to disagree, and we could have a conversation to explore or overcome that point of disagreement, but before you disagree, I need to be clear that you understand what I am saying here, first.

Yes-yes, I now understand. I just had a totally different idea of a schematic medium. I’ll describe in the next section.

As for disagreement, I do not anymore. I see no contradictions in your system now. We can proceed to compare our two approaches.

OK, here – remember, I’m talking about the difference between a “medium,” and a language.

I was talking not the types like in programming languages.

Is it OK if we remove all these comments and place a summary instead of them?

TimurIsmagilov's approach

  • No coordinate system. This medium only defines topology and contents of elements, not their location.
  • Shapes present:
    • Text shape. It cannot contain anything other than text.
    • Group shape. It can encase any number of shapes in it.
  • None of the shapes bear any semantics but they have several fields (they are not contents):
    • Type. They bear no semantics when talking about the medium but can have it when applied to languages.
      • For example, in a programming language, a text shape could have two types: string and id. It is up to editor program to decide how they should be shown; strings could be shown in proportional font, ids could be shown in monospace font.
    • Lines connect shapes and define relations between them. They have direction.

Here’s an example program: link to png image. It is equivalent to this pseudocode:

stdout.print(
  map(
    function s (
      return subst(s, "dead", "alive & kicking")
    ),
    unordered-set (
      "the world is dead",
      "wikis are dead too"
    )
  )
)

Note that I had to introduce at least one variable name s to translate the program.

Schematic source code has these advantages:

  • Can be shown in any way. In the example above, I used one way. But it could also be shown like that: todo image
  • You need less variable names, you can just point to a thing you need. Disadvantages:
  • I encourage you to write something that comes into your mind.

Take a look at link to png image. There are two programs present. Their visual representations are less verbose than in the first example. Note how in the first one it is easy to notice a loop and how the second one is divided into steps. Schematic medium allows more freedom in placement than text medium, thus kinda making parallel computations natural; thus steps can be introduced.

new: 2021-02-02 13:25 UTCLionKimbro:

I think you’ve created a language, but I don’t think it’s a medium. For one, it’s not very expressive, and number two, it’s tied to your particular programming language; it’s not generic.

I’m focused specifically on expression – expression implying form, specifics, drawing, space, positionality.

Incidentally, notice that the language you’re outlining is expressible within the schematic medium that I’m defining.

Right? You’re using boxes, text, and lines, all of which are essential elements of the medium.

new: 2021-02-02 13:25 UTCLionKimbro:

I just realized that Timur removed my text, and replaced it with a summary, that missed essential dimensions of my idea.

So, I’ve rewritten and expanded the original article, and included pictures, to see if it helps explain my idea, the central thesis of my idea.

new: 2021-02-02 13:25 UTCLionKimbro:

I need to note – neither of these is a Schematic Medium:

  • https://github.com/enso-org/enso Enso is a general-purpose programming language and environment for interactive data processing. It is a tool that spans the entire stack, going from high-level visualisation and communication to the nitty-gritty of backend services, all in a single language.

Enso is a programming language. Yes, it has an environment too, but it is specifically tied to its specific programming language.

It could be expressed in a schematic medium, but it isn’t a schematic medium.

  • https://en.wikipedia.org/wiki/DRAKON DRAKON is an algorithmic visual programming and modeling language developed within the Buran space project following ergonomic design principles. The language provides a uniform way to represent flowcharts of any complexity that are easy to read and understand.

DRAKON is also not a medium. It’s a programming language. It’s a visual programming language, too. But that doesn’t make it a medium.

Compare:

DRAKON programs could be expressed within a SchematicMedium, but DRAKON is not a SchematicMedium.

Python programs can be expressed within TextFiles?, but Python is not TextFiles?.

new: 2020-12-25 07:46 UTCTimurIsmagilov:

I think you’ve created a language, but I don’t think it’s a medium.

No, it’s a medium. It just so happened that I showed an example of using the medium with a language. Several languages, in fact.

For one, it’s not very expressive, and number two, it’s tied to your particular programming language; it’s not generic.

It’s quite expressive. I guess I’ll have to rewrite everything, just like you did. I’ll do that on a different page though. I’ll comment on your rewriting later.

Right? You’re using boxes, text, and lines, all of which are essential elements of the medium.

Yeah, sure. Because this is a medium. Maybe expressing myself verbally on WikiMeet 2021-02-06 will be easier.


Enso is a programming language.

I fully agree. Just wanted to mention it.

DRAKON is also not a medium. It’s a programming language.

I kinda agree. I don’t really consider it a programming language but it’s not a medium either.

Thoughtstorm response

[1]:

I’m afraid we’ll have to part company on your latter assertion. I love text files. There’s nothing that can’t be represented by a good text file. I’ll go further and say that it’s denial of text-files (and the underlying file-system standards, text file management tools) that ultimately killed SmallTalk. Had someone seen this clearly in 1980, and figured out how make a transparent two-way connection between Smalltalk space and text-file space, we might have seen a very different evolution of computing : a true synthesis of Smalltalk and Unix.

(CommunityWikiFooter)

Define external redirect: VisualProgrammingLanguages TextFiles

EditNearLinks: TextEditor SmallTalk PhotoShop ZoomableUserInterface

Languages: