FrontPage SiteMap RecentChanges HowTo Blog

Matching Pages:

RSS

Zimbabwe, National Day

MachineCodeBlocks

MachineCodeBlocks are a format several people are creating, an example of InterCommunityCooperation.

The format is for adding meta-data to wiki pages.

The people presently tinkering on it are:

This MetaData format is special because:

The format is currently in specification.

Examples of Core Ideas

While the format is still in specifications, we know enough to give some examples that demonstrate our core ideas.

Three Ways to Say the Same Thing

On the page LionKimbro, you might find the following metadata:

Notice:

Here’s another way to represent it:

MACHINECODEBLOCK ;
name:Lion Kimbro ;
namepage:LionKimbro ;
memberof:CommunityWiki ;
memberof:OneBigSoup ;
homepage:http://www.speakeasy.org/~lion/ ;
blog:Lion's Den ;
FOAF:http://www.speakeasy.org/~lion/foaf.rdf ;
MACHINECODEBLOCK ;

This parses to exactly the same thing. Again: Remember: All HTML is ignored, and all hyperlinks are the value of their link target, not the text of the link.

Of course, this is legal as well:

 MACHINECODEBLOCK ;
 name: Lion Kimbro ;
 namepage: http://emacswiki.org/cw/LionKimbro ;
 memberof: http://emacswiki.org/cw/CommunityWiki ;
 memberof: http://emacswiki.org/cw/OneBigSoup ;
 homepage: http://www.speakeasy.org/~lion/ ;
 blog: http://lion.taoriver.net/ ;
 FOAF: http://www.speakeasy.org/~lion/foaf.rdf ;
 MACHINECODEBLOCK ;

Those three expressions compute to the same thing.

We care about multiple ways to express things, because we want to support all media.

They should all be able to hook up with each other and link with each other.

Example Uses

Baseball wiki. Each baseball player and team would have a page devoted to them. Each page contains human-readable discussion as well as a MachineCodeBlock? to give machine-readable player/team stats. See RdfForWiki for similar ideas.

Internet Communities. Make descriptions of OnlineCommunities, the membership of online communities, and the technologies that they use. We can then point our descriptions at one another, and build a graph to graph our relationships.

We want to describe groups, their members, and affiliations between groups. This may aid our InterCommunityCooperation efforts! The theory is that when we can see each other, we will be able to identify our CommonCause and the shape of our aggregate relationships. This may encourage communication and cooperation; see SocialNetwork.

Code

We have a Python interpreter for an older MachineCodeBlocksInterpreter. We also have some WikiFormats code somewhere as well.

Meetings

We presently meet haphazardly, as time allows.

The MachineCodeBlocks format owes a lot to the WikiTing sessions.

Here are our meeting notes so far:

Other Ideas

We’re currently trying to work out:

Alternatives to MachineCodeBlocks:

Notes in Holding for Next Meeting

At the moment, in SdiDesk, my page-variables use a #= assignment operator eg. a line which contains
 x #= 42 
will add a variable x with a field 42 to the dictionary. But the convention is not established so I could change it.

What’s the latest thinking here?

PhilJones

You could use WikiVirtualData format:

 * x: 42
 * x= 42

which doesn’t clutter pages. We use it widely. This means things like

are already valid data entries. Configuration metadata is held on subpages:

which makes our wikis very flexible.

HelmutLeitner

Two of our major design requirements are:

  • It must work on all WikiEngines.
  • It must support linking.

By linking, I mean: You have to be able to take one cluster of data, and express that there is some relationship between it, and another cluster of data found on another page.

We also don’t want to have to manually write out URLs. That is, we designed it to piggyback on the linking mechanisms that are already found in wiki, blogs, etc.,.

The InterLink, the NearLink, whatever- it should be possible to use them when forming links.

So, in the MachineCodeBlocks system, we (generally) favore <a href=”…”> over the actual linked text.

It’s more complicated than this, as the system stands, but that’s the basic idea.

(I’ll move this conversation out to a discussion section, if nobody objects.)

I think it is important to rephrase the differences between the current proposal discussed on this page and the ideas mentioned by MurrayAltheim, HelmutLeitner, and PhilJones. Here’s how I understand it. The rephrasing may help people check whether their ideas came across as intended or not. (As typically I never quite understand Lion’s ideas the way he intended them… ;))

  1. WikiVirtualData is a special case for existing TextFormattingRules. The raw text is parsed by the wiki engine itself to generate the HTML sent to clients, and it is also parsed by other pieces of code that want the actual metadata. In this, wiki virtual data is explicit metadata that appears in the text itself.
  2. MachineCodeBlocks are defined very loosely so that the information contained can be extracted from the raw text and from the HTML itself. Machine code blocks no longer require a particular raw text to be intelligible, they are independent of text formatting rules, and the specification can be used be weblogs as well as any other media accessible via an URL. The machine code blocks don’t even have to be visible to the end user where as wiki virtual data usually is. This explains why the machine code block definitions needs to account for all sorts of “fluff” – we need to specify how to identify the keys and values in any kind of data stream, be it wiki text, HTML, XML, or anything else.

For (2) some XML syntax should be defined, but I don’t see one. (2) could be created from (1) too, so the concepts are not so far apart. What’s hardest for me as a language aficionado is the term MACHINECODEBLOCK, because it is neither about a machine, nor is it code but data, nor has it anything to do with machine code (native processor code).

Alex, that’s a good description. But it raises the obvious question as to why there needs to be two different machine-readable data formats with different characteristics. I understand that some wikis already have a data-format and will need to keep supporting it for legacy reasons, but for new development, I’m not sure I see the point of this distinction between WikiVirtualData and MachineCodeBlocks. It seems that MCBs are essentially hidden from users but support auto-discovery by other applications that are reading the wiki. WVD on the other hand, isn’t hidden.

How important is this difference? And couldn’t the hidden-ness of the MCB be a user-decided, formatting option? For example, SdiDesk has a <hide> tag for things you don’t want to appear in the presentation view.

I also sense a confusion about who is “producing” the MCB? Is it a user, editing the page? Or is there a hint that maybe the wiki will produce it automatically in the HTML version but that it might not appear in the raw text?

Final question. I’m no great fan of RDF. But SemWeb? people will be accusing you of re-inventing it. Did you consider some suitably wiki-like representation of RDF triples (such as N3) as the format? And if you rejected it, why?

PhilJones

Lots to respond to here. I’ll start with PhilJones. I don’t have time to respond to everybody and everything at this moment.

  • MCB’s are almost always visible to users. (“ugly”)
    • This is partly because we take advantage of Wiki’s WikiWord-to-URL rendering capability.
    • ESW:NetworkedData is particularly important to us. Linking must be easy, and it cannot require writing out URLs. We think an OnlineCommunity should be able to use the URL rendering capabilities available to it, be it WikiWord, InterWikiLink?, NearLink, LocalNames, whatever.
  • Users write MCB’s.
    • Though, it’s conceivable that a WikiEngine might write them out automatically.

The most important question is about RDF and MCB’s. I want to answer, but I don’t have the time right now. I’ll get back to it later today.

Lion says that MCBs are normally visible to users. In which case, what’s the difference with WikiVirtualData and why do you need two different notations for meta-data?

PhilJones

Briefly, the major differences justifying different notations are:

ESW:NetworkedData is imporant; Data is more valuable when it can be linked.

Other points:

  • Yes, other wiki implement their own MetaData systems. This is good. One day, there will be a network of translation engines. They will translate from system to system. RDF, TopicMaps, MachineCodeBlocks, they will all connect with one another.
  • I do not worry that SemanticWeb people will accuse me of re-inventing RDF. That is okay with me.
  • I’ve considered ESW:RdfXmlSyntax, ESW:NotationThree, Turtle, and other ESW:ShorthandRDF systems. MurrayAltheim has exposed me to TopicMaps, too. I reject these, because they are too complicated for regular users, and because I don’t need or want the power of RDF in these situations. Using a firehose to clean the kitchen sink, and all.

I believe people would like to sit down, and comfortably write out:

MACHINECODEBLOCK-BEGINS ;

Name:Joe Demaggio ;
Team:BostonRedSox? ;
League:AmericanLeague? ;
Nickname:The Yankee Clipper ;
Wikipedia-Page:WikiPedia:Joe_Dimaggio ;
Lifetime-Batting-Average:.325 ;
Born:1914-11-25 ;
Died:1999-03-08 ;
Photo:http://en.wikipedia.org/wiki/Joe_Dimaggio ;

MACHINECODEBLOCK-ENDS ;

I don’t believe most people need to be thinking about ontology, type, schema, RDF, RDFS, OWL, or graph merging.

For the most part, I (& others) just want to:

  • feed data (strings, mostly) into a computer,
  • hook the data up with other data.

Have I answered your questions? Do you have any other questions?

Helmut, I am sympathetic to your arguments.

But, I think we’re past the point of no-return, namewise. I don’t know how it happens, but there comes some point where the name for a thing has a life of it’s own. Observe HiveMind, HiveMindName. I’d love to call it TransMind. But it just wouldn’t work, now. It would take a lot of concerted effort to change it now.

As for “code”- consider: “morse code?” Surely they are not all writing programs in morse code. {:D}=

I was thinking, “It’s a block, and it encodes data so that machines can read it easily.” That’s what I was thinking.

But I understand and am sympathetic to your arguments.

If I could mindwipe everybody, and recode their understanding, I would maybe call it “Visible HTML Data Blocks,” or “Easily Produced Data Blocks” or “Web Data Blocks” or something like that.


Phil says : I’m a little sceptical of your claim that MCBs are really universalizable across wikis. For example, how does the MCB reader parse the HTML produced? I can imagine situations where a certian HTMLizition of an MCB introduced new characters which broke any standard parser. For example, if it added new-line characters, invisible in HTML, so that the parser would fail to see the connection between “Name” and “Joe Demaggio”. I don’t believe that this is an ideal you can aspire to. In which case, we fall back on a “raw” standard, or at least a family of acceptable raw standards.

PhilJones

how it works

The interpretation of the HTML is a little complicated, because there are ways to say (by our present spec): “Take the raw HTML produced here.” It’s also a little complicated, because hyperlinks are treated specially.

But for the most part, it works like so: Huck all HTML tags. Decode all entities. Interpret the remaining text.

why it is universal to all wiki

Because MCB’s can be written in an amazingly broad variety of ways. This isn’t like most specs, where there is one, or a very limited, way of writing things. This spec gives you a big number of ways to do it.

Stray newlines are not a problem, for a few reasons.

  1. We’re not seeking perfection. This is super-pragmatic. (Perfection has a place in the world, this isn’t that place.)
  2. The worse that can happen is that an extra newline appears in the string value.
  3. I don’t know of a single wiki that insert newlines in the output HTML.

A newline between “name : Joe Demaggio ;” can only, at worst, confuse the rendering of the value “Joe Demaggio.” Parsing is extremely liberal. You can have all sorts of junk before the key, and it will all be ignored.

If you had: fibble fibble foobar fop name:Joe Demaggio;freep flooble, …

That is interpreted as just:

  • key: “name”
  • value: “Joe Demaggio”

So, you see, we do not care about raw, and are portable to every single wiki in the world.

Pretty cool, eh?

See Also

Discussion

Development mostly happens in haphazard live sessions, sparked in the WikiTing or #onebigsoup.

If you want to talk on this page, that’s okay, but recognize that this is an InterCommunityCooperation, and that there are no dedicated channels for this yet.

You may, however, want to attach a note in the “Notes in Holding for Next Meeting.” When next we meet, we may (if we think to) check that list for things to bring up.

I’m surprised to see no discussion of YAML as an alternative. It seems to do the same thing in a broader context, and already has a fair amount of support.

The reason we need MachineCodeBlocks is because:

  • it will work on all existing wiki (and also blogs)
  • it can link outside of itself (NetworkedData?)

YAML’s a different field entirely. 1 A more valid criticism would be that MachineCodeBlocks are not RDF. But RDF isn’t supported by the vast majority of wiki.

Why is NetworkedData? an essential feature? Imagine if HTML didn’t feature the ability to link to other web pages. Imagine if, instead, web pages could only link to other parts of themselves. A web page could have a table of contents linking to itself, say, but you couldn’t link to any other web pages. Just imagine: How stupid would web pages be, if they couldn’t link to one another?

And yet, that’s how the vast majority of data formats work! The vast majority of data formats feature internal linking, but not external linking.

YAML has mechanisms for internal linking, but not external linking.

But more than that, you can’t even express YAML in most wiki. Perhaps someone, somewhere, has made a wiki that can express YAML. I haven’t seen it, though. And if there was one, or two, I don’t think that it’s going to become near-ubiquitous.

We can have near total ubiquity, with MachineCodeBlocks.

That’s why.

Does this answer your question?

I’m also looking at JSON, YAML, micro-formats etc.

It seems that both the ProamInternetCommunicationsAlliance and the microformats wiki are looking at the same sorts of things. Should we be talking about machine code blocks on those two wiki? Or at least link those wiki with WikiNode?

Yet another early “simple, human-readable protocol”: “Simple Protocol Application Data Encoding” SPADE 2000

Footnotes:

1. I’ve participated in YAML groups, used it myself, and see Mike Orr regularly. I’m a great fan of YAML, and wish people used it more often, when they instead use XML.

Define external redirect: MachineCode NetworkedData PageVariables AmericanLeague InterWikiLink TemplatePage BostonRedSox MachineCodeBlock SemWeb

EditNearLinks: OnlineCommunity WikiEngines FlorianFesti SdiDesk WikiWord WikiEngine OnlineCommunities WikiVirtualData

Languages: