FrontPage SiteMap RecentChanges HowTo Blog

Matching Pages:

RSS

Peru, Independence Day

CommunityProgrammableWiki

A CommunityProgrammableWiki is a wiki in which the code of the wiki itself is collaboratively edited by its users.

Why?

First, it is always interesting to take concepts to their logical conclusion. The fundamental principal of wiki is that everyone in the user community has equal control over every part of the site. In conventional wikis, however, the webmaster still has exclusive access to the site’s underlying software. It will be interesting in itself to explore what happens when the principal of community control is applied more broadly.

Second, many who have analyzed the social dynamics of wikis conclude that, in the context of a wiki collaboration model, powerful individuals are harmful to the community1. Hence, for the sake of the community, it may be helpful to take the exclusive power of control of the software away from the webmaster and distribute it to the community2.

Third, often a wiki’s host does not have the time to upgrade the software. With a C.P.W., that duty can be distributed to a large number of people rather than sitting on one person’s shoulders.

Fourth, the collaborative model of wikis has proved very effective for creating textual content. If the same model works for software development (where the content is computer code rather than human-language text), it could prove to be more efficient than current collaboration models. There is already evidence that greater decentralization is better for software development3 4

In other words, CPW should enable wikis to become more up-to-date and for newly available wiki features to more quickly be adopted by wiki web sites, while simultaneously decreasing the workload of the site administrator. In the best case, it is also possible that CPW will enable a new collaborative software development paradigm that may accelerate the pace of software development, including the development of software that has nothing to do with wikis.

See also CommunityProgrammableWikiMotivations.

Implementations

There is a prototype implementation; see CommunityProgrammableWiki:CpwTutorial? for a quick runthrough.

KennethTyler? of SeedWiki is working on a a CPW too.

See also CommunityProgrammableWikiImplementations.

Further discussion

Should be relegated to other pages, for example CommunityProgrammableWikiIssues, in order to keep this page managable.


Related pages


CategoryUncommonWikiFeature? CategoryWikiTechnology CategoryWikiEngine

Discussion

I think we can do it, and I think it’d be cool.

I was thinking: “Gee, this really sucks, because you wouldn’t know what the code did, how it worked.”

Then I realized: “Wait, I could communicate with fellow developers inside comments.

That is, it’d be a whole new way of learning about code: Communication within comments.

It’d be a social experience around the code, like in the DevelopersVirtualWorld.

It’d be a social experience around the code, like in the DevelopersVirtualWorld.

Yeah!

I am also interested on the effects that something similar to WikiProcess would have on a coding project. I think that would make the experience more “social” (and more efficient), too.

And another thing--

Epiphany time--

You’re not talking about a CommunityProgrammableWiki, what you’re really talking about here, is a CommunityProgrammableInterface?.

That is, it’s a GroupServer, SharedAwarenessSystem, thing, that is community programmable.

That it’s wiki at the bootstrap, gives us some nice HyperCard like pleasantries, and convenient data handling and code storing and documentation thingies, and some SoftSecurity as well.

But, really, it’s a CommunityProgrammableInterface?.

And hence the vision of the WikiGateway, and stuff, as well, as attachment.

I’m seeing it, I’m seeing it.

I get it; I’m being pulled into the attractor.

Yes! Exactly! I don’t talk about that so much because it’s kind of pie-in-the-sky and I don’t want to scare people off with vague visions. But in the best case, the wiki is just the bootstrap, and the programming system that is developed can be used for anything. In a way, though, it might still be wiki, because the social process would be similar to WikiProcess.

What Programming Language?

Hi, I can’t seem to find the page where you said it, but I thought I remember you suggesting Ruby or Io as a language for a CommunityProgrammableWiki?

Here is my current thinking on language:

  • My current favorite language in general in Python
  • A handful of people have suggested Ruby for a CommunityProgrammableWiki, so maybe that would be a better choice for this particular project, though. I don’t know Ruby yet myself.
  • However, I would like a CommunityProgrammableWiki frontend to CommunityWiki – so for interoperability reasons, it may be wise to fork OddMuse, which means using Perl (at least initially – once the project gets going it would be possible to slowly, incrementally rewrite the code in another language)
  • I read TypeZ:IoLanguage. Io looks cool – I’ve been looking for a language with simpler and deeper reflexivity than Python. But I don’t really want to deal with a language which doesn’t have a large set of libraries, given that a CommunityProgrammableWiki will be on the web and will use various standard formats and protocols.
  • People have also suggested taking a look at Scheme, although maybe they just meant “for inspiration”.

All things considered, I am strongly leaning towards forking OddMuse - I am just really excited to use the CommunityProgrammableWiki in my interaction with the pages on CommunityWiki. We could write some scripts to keep patching our fork with the latest changes to mainstream OddMuse, so that it wouldn’t be a fork so much as a patched version of OddMuse.

Perhaps later, we would incrementally rewrite OddMuse in another language. The “other language later” would probably be either Python (since Lion and I both know it) or Ruby (if we get around to learning it, and if it’s as cool/appropriate as people say).

Also, you linked to LesserWiki? above – do you intend to make LesserWiki? into a CommunityProgrammableWiki?

> Python, IO, Ruby, Scheme, perl, etc.

I’m easily impressed with the flying divs of LesserWiki? but prefer the http://wiki.wikiwyg.net inline editor.

My language experience is mostly C/C++ application development, though I have dabbled in driver writing on both 95 and NT/XP, and recently studying Linux kernel devl.

I’ve played with Scheme off and on, trying to “make it work”, but never had much success. The ‘looseness’ of Lisp makes it very fun, but I’m a novice there too.

The largest concern I have with programmability is security, but the second is ease.

It is for ‘ease’ that I am considering these other languages. While the ‘exposed’ needn’t be the same as the implementation language, it’s easier that way…

I have more to write about how workers might earn ‘change’ power through peer ‘accept’ ‘reject’ of his patches…

And how we might use simple rotating ‘quizes’ for light security - to at least stop bots, but to also determine if the worker is not sincere…

I recommended Scheme, because it’s trivial to sandbox, and there’s an implementation of Scheme in practically every language. Here’s pyscheme. pylisp. Whatever. :)

The neat thing about doing this in something that’s sandboxed, is that you can now make commits immediately applicable. You could have a test server, and the main server, and a button to reset the test server back to a stable version. People could start making edits on the test server, and then when they’re ready, submit the big commit instruction to the main server. If nobody hits the panic button within a day, it’s done.

If the JVM is adequately sandboxable, (say: “Can’t make outgoing network calls, except as approved by sysadmin, can’t use more than 30 megabytes of disk space, can’t blah blah blah,”) then we could use Jython as well. I mean Java. That’s what I mean- Java.

I’d go with Python. We know it. There’s nothing cool in Ruby that Python hasn’t already had for 5 years, and my take is that the Ruby syntax is harder/trickier than Python’s.

I wouldn’t use any “interesting, deeply fascinating, strange” smaller languages, unless I was studying languages.

CommunityProgrammableInterface? makes me wonder about even less power.

CommunityChangableInterface? - where more of the site’s ‘look’ can be adjusted, but no ‘actions’ can be altered…

Trying to understand…

Rough divisions of authority:

Owner: Pay hardware and hosting costs. Currently the only sovereign actor.
Steward: Lightly steer activity toward long-term goals of community
Hacker: Playful cleverness
Customer:
Community:
Vandal: Malicious

Rough divisions of power:

Read Text:
Read Code:
Write Text:
Write Code:
Commit: Some changes may only be committed by agreement?
Rollback:
Change Interface:
Change Action (code):

(sorry, not a response, just an idea I’m dropping off- actually have a response, but it will come later)

It occurs to me that the “community programmable wiki” doesn’t need to come all at once.

You could start by making some things community programmable.

Include a little scheme interpreter, and then convert some functionality to run based on the scheme stuff. Gradually put more functionality into the “community programmable” side of things.

Languages

Lion, you said:

I recommended Scheme, because it’s trivial to sandbox, and there’s an implementation of Scheme in practically every language.

I’d go with Python. We know it.

So which do you actually want, Python or Scheme? :) Or Scheme within Python?

You could start by making some things community programmable.

Include a little scheme interpreter, and then convert some functionality to run based on the scheme stuff. Gradually put more functionality into the “community programmable” side of things.

I’d rather do the reverse:

  1. Start out using an extent wiki like OddMuse; make everything programmable
  2. Then, if we want to switch to Scheme, add the Scheme interpreter and incrementally rewrite the wiki in Scheme

(or we could start with a wiki written in Scheme and just use that; but that would make it harder to use this as our front end to CommunityWiki)

CommunityChangableInterface? vs. CommunityProgrammableInterface?

Btw, I’ve been referring to anything less than a fully reprogrammable wiki as an ExtensibleWiki? (not saying that we have to use that language, just telling you in case you want to look at the ExtensibleWikis page).

Security vs. ease

Lion, I would love to allow users to execute immediately on a “test wiki”, but I just don’t think it’s secure enough. The sandboxing has to be 100% secure, with no mistakes, for this to work. In real life, there will probably eventually be some flaw discovered in the sandbox, which would allow an attacker to instantly take over the server and get the host in trouble by making the server do evil things. I really think it’s necessary for the community to review any proposed code before it runs on the server, even given sandboxing.

Now, I can think of 2 ways to ameliorate this annoyance:

  • Just like we have CommunityWikiAdministrators here, we could have a HardSecurity-enforced class of admins on CommunityProgrammableWiki who have the privilage of being able to commit code immediately on the server.
  • We could make it really really easy for anyone to download, run, and edit an instance of the wiki on their own machine, for them to edit that instance, and for a seamless transition between using that instance and using the server instance (that is, as much as possible, it auto-installs, auto-configures, redirects you to the correct page on the other instance, etc – the instance on your personal machine would KNOW that it is a debugging instance and would be in communication with the instance on the server – when you are done debugging your new code, there is an easy way to upload the change as a proposed code change).

Both of these will take some implementation work and so won’t be available at the beginning of the project (until we get around to implementing them).

I’m not saying I won’t participate in a project that doesn’t meet my security criterion (that is, there has to be code review BEFORE untrusted code is run on the server), but I am not willing to host such a thing (or to ask someone else who is not involved in the project to host it).

So here is my proposal for how to proceed:

(step 0: find time to do all this; finish any prerequisite projects)

  1. Locate a hosting service. The best bet that I know of us Eugene Kim’s offer to host us. Other ideas include asking SourceForge to make an exception in their terms of service for us, asking Alex to host us here, or asking Lion to host us.
  2. Take OddMuse. Replace all of the “backend” subroutines, those which access the PageDatabase, with subroutines that use WikiGateway to interface with CommunityWiki. At this point, we have a CGI script that provides an OddMuse frontend to CommunityWiki.
  3. Make the wiki programmable by adding a script action that searches for pages containing a specially identified “patch”, and applies that patch to itself if and only if no one has changed that page for 1 week AND if one of the admins has approved it for security. I’ve already written a similar thing for UseMod, it was pretty short (but it didn’t check for admin security approval)
  4. At this point we have a very very crude programmable frontend to CommunityWiki. Do everything else within the system.

There are surely a million things we’ll want to do after this to make it easier to use, to make a more usable “parliamentary procedure” for us to review and approve code suggestions, to rewrite it in another language that supports sandboxing, to make a “backup server” in case of an error, to integrate with version control , to support branches, to do literate programming i.e. viewing the code, including versioning, as wiki pages rather than just having “patches” on wiki pages, federated CommunityProgrammableWiki servers exchanging code patches, etc.

But this way we can begin to work to “within the system” as soon as possible And we can “scratch our itches” in terms of adding feature useful to us as CommunityWiki participants.

I recently bought one year hosting and DNS reg. at http://epfarms.net for $32.76

Hosting is $2.00 monthly
Domain name registration is $7.50 yearly

I’d love to be part owner in such a venture.

Thanks Patrick! OK, we can add that to our list. I’m kinda worried about commercial ISPs, though, since we may need weird stuff running (like a version control repository), or installed (like Python, Java, Jython, Ruby, Scheme, and various library modules for whichever) or may want to do OS-level sandboxing (chroot jail) . On the other hand, I know some hosts give you your own box (or virtual box) and let you administer everything, including the OS. What do you think?

Also, I forgot to mention that I could host on my work computer. I can’t guarantee 24-7 access (in fact, we are regularly offline on wednesday evenings; also, the lab is going under construction soon, so there may be a week or whatever when i dont have a connection at all), but realistically, it’s an o.k. option.

Also, I forgot to say: so out of the languages that have been mentioned above, the ones that support sandboxing are java, scheme, ruby, and presumably jython, right?

No skills over here to help, but wanted to tell ya’ll I was reading along and think it’s really cool!

Talk of ownership makes me nervous, I’d much rather just say: “The code is PD, and operational concerns are by consensus.” But perhaps that’s just me.

The costs are so negligible, I’d be happy to foot the bill for the entire thing.

I’m also totally okay with granting shell accounts w/ SSH on my computer.

That said, my server isn’t totally reliable, since there’s a weak power cord leading to the hub.. =^_^=

> host on my own

That makes me wonder about a P2PWiki - is that already done? That would be great.

> No skills

What is a safe and fair way to collect small amounts of money for hardware and toll costs? We need an at-cost FreePal? service.

Moved ‘ownership’ discussion to WhatIsMoney.

Here’s a radically simple Python scheme implementation.

It should be easy to sandbox: Only give it capabilities through very specific boxes.

So for example, create a built-in for pulling down a web page, and stick a bunch of throttlers and automated logging on top of it.

I opt for using JavaScript as language.

Reasons: if the audience is really the public: use the same language for client side programming (firefox) and for the server.

Random thougts:

  • collaborate with TiddlyWiki, this is already a community programmed wiki ( but without a wiki workplace)
  • try to do as much of the computation on the client side (to develop to a p2p wiki)
  • using javascript you get a serverless programming environment
  • implement a DataServer which stores data local but is able to fetch data from other wiki instances
  • DataServer might be programmed in python, implementing a very narrow feature-set, every other tasks could be done in the user-interface, so : no server
  • using a “in the browser implementation” the browsers is the sandbox: no need to care about a server
  • http://webcrossing.com/ has a web-server with integrated oo storage , programmable by javascript

Another “brainstorm”: it should be quite easy to hack some javascript-support (like the CommunityMaintainedCss) into OddMuse: extracting parts of pages as javascript, javascript-includes. OddMuse could provide WikiProxy functionality by implementing a RPC-interface, returning JSON

This JavaScriptEnabledWiki? could evolve to a JavaScript CodeSnippCollection? with function to parse HumanInput? , CreatingForms? , ExplainingAlgorithms? , providing ExampleDatasets? etc …

Yes! You beat me to it!

I had just thought the same thing myself, just the other day- “Use JavaScript on the client!”

So, you edit the JavaScript, and that’s the communal code, and it’s all sandboxed in the user’s browser…

If there were some “horrible capability,” it would have already been done by now.

BIT: “Yes-yes-yes-yes-yes-yes-yes.”

Bias warning: I’m naturally resistant to this idea because I don’t know JavaScript, and my few experiences with it have not been great.

Hmm.. I am very much in favor of breaking wiki software down into smaller modules, and moving processing to the client side when possible (see ModularWiki, also section 2.4 of my 2005 WikiGateway paper for places where I’ve argued this).

But this alone doesn’t argue for JavaScript, right? You could have client-side processing and server-side processing done in any two languages, communicating through some protocol.

The arguments for JavaScript are:

  • Client-side:
    • interoperate well with the web browser
    • sandboxable
  • Server-side: ThomasKalka put it best: if you’re using JavaScript on the client, may as well “use the same language for client side programming and for the server.”

I am still undecided. These two seem like strong arguments. However, for me personally, my (unfounded) dislike of JavaScript is also strong.

Another consideration in the short term is that I think it would be best to start with Perl so that we start with OddMuse and then incrementally change it, all the while using our code as an alternative frontend to CommunityWiki (a ProxyWiki?/WikiWindow). This doesn’t rule out using JavaScript or Scheme or Python or whatever eventually; we could incrementally rewrite OddMuse into whatever language we want.

However if you wanted to start with TiddlyWiki instead of CommunityWiki then I guess JavaScript would be a good choice from the get-go.

I don’t think it really matters so much what the server is implemented in; I suspect you only need a public, regular interface.

I suspect 95% of the code will be in the client, and only 5% on the server.\

The server would basically just be a large data warehouse, no? In fact, a DataServer?

Since it’s the client doing all of the formating and what not.

I believe this line of development will be very productive. A few Observations that support this opinion…

  • Eric Shulman is one of the major developers of a set of “Adaptations” that build on TiddlyWiki.
    • He has built a great set of Import & Export utilities that make it relatively trivial to move the Tiddler content between files.
    • He is working on a MySql? backend for this software as well as incoreporating many ohter really neat features. One of these is what he calls “Sliders” that are revealed via an apparent page expansion, onMouseOver.
      • I’ve been extending this concept, since I find it very useful and have used the onMouseOver to spawn a sqlServer query (a la Google Suggest) that spaws either a pop-up note or another window, both of which can contain HTML (or wiki markup) that can be nested.
        • To visualize this, think of Flickr’s Tag boxes, except that the Tags are hyperlinks that can be activate via a mouseOver event. With this approach, it is even possible to use Font Size & Color to distinquish the Tags, effectivly creating a Set that is capable of containing many possible Types of Tags. (And, of cource, it’s infinitely expanadable!)
  • Since there is quite a strong group of developers that are each creating their own “adaptations” like Eric’s, on top of the Core TiddlyWiki that Jeremy Ruston develops, this effectively allows a “usre” to pick and choose the javaScript User Interface modules of their personal choice, and pull data from a real, industrial strength, database for the shared content.

I’ve been spending quite a bit of time of this stuff since Christmas, and I am quite confident that this approach is very worth-while.

Wiky is a clientside Wiki markup to HTML converter written in javascript. As it is bidirectional, it can convert Wiki markup to HTML and later convert that generated HTML text back to Wiki markup.”

Say I have Wiky on the client side, and one of the file systems mentioned in the “distributed-replication software” section of WikiFeatures:FailSafeWiki as “the” server.

What else do I need to do to get this to work?

I think:

Versioning (on the server-side, and additions to the clientside to work with it). LocalNames. RecentChanges.

And, at some point, server-side spam protection.

Note that (I think) WikiWyg requires a “Wiky”-like component for each particular WikiMarkup? (i.e. in order to integrate WikiWyg into a given WikiEngine, you must provide both directions of translation – I think both directions must be in JavaScript).

Note also that the InfiniteMonkey wiki engine will have a JavaScript markup module.

I was pleasantly surprised to see this. Next, I’m prepared to be “amazed” when it turns out that there is actually some coordination of these efforts. :-)

The site http://92.243.19.149 is a programmable wiki. In fact, you may want to read this page : http://92.243.19.149/cgi-bin/bluemoon/wiki.pl?action=lire&theuth=69&page=programmable+wiki&zone=wikis I am open to your suggestions (what should be developed first?)

Footnotes:

3. Eric Raymond, 1998. “The Cathedral and the Bazaar”, in First Monday, volume 3, number 3 (March). At http://www.firstmonday.dk/issues/issue3_3/raymond/
4. It must be noted, however, that software is essentially different from natural human languages. For example, if one page of an English document is changed drastically, the rest of the document will probably still be understandable. However, if even one line of a software program contains an error, the entire program may fail to run. This particular problem may be ameliorated by unit testing, peer review, and a backup wiki. But because of this and other differences, any extrapolation about the suitability of wikis for software development based only upon its performance with textual content is only a guess.

Define external redirect: CreatingForms CpwTutorial LesserWiki CategoryUncommonWikiFeature WikiMarkup CommunityProgrammableInterface FreePal ExtensibleWiki ExplainingAlgorithms HumanInput CodeSnippCollection CommunityChangableInterface JavaScriptEnabledWiki KennethTyler SkWiki MySql ProxyWiki ExampleDatasets

EditNearLinks: PageDatabase MetaWiki WikiWyg CategoryWikiEngine PublicScript ExtensibleWikis UseMod HyperCard CommunityProgrammableWikiIssues SourceForge WikiEngine HardSecurity CommunityProgrammableWikiMotivations JavaScript SeedWiki

Languages:

The same page on other sites:
MeatBall:CommunityProgrammableWiki