"I think that conversations are the best, biggest thing that free software has to offer its users"

With: Alexandre Leray, Andreas Vox, Asheesh Laroia, Carla Bosermans, Cornelia Sollfrank, Chris Lilley, Christoph Haag, Claire Williams, Claudia Krummenacher, Denis Jacquery, Eleanor Greenhalgh, Eric Schrijver, Evan Roth, Femke Snelting, George Williams, Gijs de Heij, Harrisson, John Haltiwanger, Juliane De Moerlooze, Julien Deswaef, Ludivine Loiseau, Matthew Fuller, Michael Murtaugh, Michael Terry, Miguel xxxxx, Pedro Amado, Pierre Huyghebaert, Pierre Marchand, Ricardo Lafuente, Stéphanie Vilayphiou, Tom Lechner, Urantsetseg Ulziikhuu, Xavier Klein, Peter Westenberg, Michele Walther, momo3010, Ivan Monroy Lopez, Nicolas Malevé, Manuel Schmalstieg
Vocabulary, community, politics
Evan Roth
Femke Snelting
John Haltiwanger
Peter Westenberg
Stéphanie Vilayphiou
Tying the story to data
In the summer of 2010, Constant commissioned artist and researcher Evan Roth to develop a work of his choice, and to make the development process available in some way. He decided to use a part of his fee as prize-money for The GML-Recorder Challenge, inviting makers to propose an open source device ‘that can unobtrusively record graffiti motion data during a graffiti writer’s normal practice in the city’. In three interviews that took place in Brussels and Paris within a period of one and a half years, we spoke about the collaborative powers of the GML-standard, about contact points between hacker and graffiti-cultures and the granularity of gesture. The text is based on conversations between Evan Roth (ER) and Femke Snelting (FS), Peter Westenberg (PW), Michele Walther (MW), Stéphanie Villayphiou (SV), John Haltiwanger (JH) and momo3010.

Brussels, July 2010

So what should we talk about?

Can you explain what GML stands for?

GML stands for [1]Graffiti Markup Language[1] [2]Graffiti Markup Language (.gml) is a universal, XML based, open file format designed to store graffiti motion data (x and y coordinates and time). The format is designed to maximize readability and ease of implementation, even for hobbyist programmers, artists and graffiti writers. Link designed for amateur programmers. It is a way to store graffiti motion data. I started working with graffiti writers, combining graffiti and technology back in New York, in 2003. In graduate school, my thesis was on graffiti analysis, and writing software that could capture their gestures, to archive motion data from graffiti-writers. Back than I was saving the data in an x-y-time array, I was calling them .graph files and I sensed there was something interesting about the data, the visualization of motion data but I had never opened up the project at that time. About a year ago I released the second part of the project, of which the source code was open but the dataset wasn't. In conversation with a friend of mine named Theo[2] [3]Theo Watson Link project[3] [4]In its simplest form, L.A.S.E.R. Tag is a camera and laptop setup, tracking a green laser point across the face of a building and generating graphics based on the laser's position which then get projected back onto the same building with a high power projector. Link .graph file again and how we could bring back the file format as a way to connect all these different applications. Graffiti Analysis[4] [5]Graffiti Analysis is a digital graffiti blackbook designed for documenting more than just ink. http://graffitianalysis.com}, L.A.S.E.R. Tag, EyeWriter[5] [6]The EyeWriter is a low-cast eyetracking system originally designed for paralyzed graffiti artist TEMPT. The EyeWriter system uses inexpensive cameras and open-source computer vision software to track the wearer's eye movements. [7]Theo Watson, Chris" target="_blank">Link Sugrue[6] [8]Chris Sugrue Link and Jamie Wilkinson[7] [9]Jamie Wilkinson [10]Graffiti Markup" target="_blank">Link Language. It is a simple set of guidelines, basically an XML file format that saves x-y-time data but does it in a way that is very specifically related to graffiti so there's a drip tag and there's tags related to the size of the brush and to how many strokes you have: is it one stroke or two strokes or three strokes. The main idea is: How do you archive the motion of graffiti and not just the way graffiti looks. There are a lot of people photographing graffiti, making documentaries etc. but there hasn't been a way to archive graffiti in ways of code yet.

What do you mean, _archive in terms of code_?

There hasn't been a programmatic way to archive graffiti. So this is like taking a gesture and trying to boil it down to a set of coordinate points that people can either upload or download. It is a sort of midpoint between writers and hackers. Graffiti-writers can download the software and have how-to guides for how to do this, they can digitize their tags and upload it to an open database. The 000000book-site[8] [11]Link Black Book: _A black book is a graffiti artist's sketchbook. Often used to sketch out and plan potential graffiti, and to collect tags from other writers. It is a writer's most valuable property, containing all or a majority of the person's sketches and pieces. A writer's sketchbook is carefully guarded from the police and other authorities, as it can be used as material evidence in a graffiti vandalism case and link a writer to previous illicit works._ } hosts all this data and some people are writing software for this.

So there are three parts: the GML-standard, software to record and play and than there is the data itself -- all of it is open in some way. Could you go through each of them and talk about how they produce uploads and downloads?

Right. It starts with Graffiti Analysis. It is software written in C++ using OpenFrameworks, an open source platform designed by artists for visual applications. Right now you can download the recorder app and from that you can generate your own GML-files. And from there you can upload these files into the playback app. In the beginning that was the only open source side of the project. Programmers could also make new applications based on the software, which also happened. Last night we met Stéphane Buellet[9] [12]Stéphane Buellet, Camera Linea Link developing a calligraphy analysis project and he used Graffiti Analysis as a starting point. I find it exciting when that happens but more often people take the file-format as a starting point, and use it as a jumping-off point for making their own work. Second was the database. We had this file-format that we loosely defined. I worked with Jamie to develop the 000000book site. It is pretty nuts-and-bolts but you can click 'upload' and click on your own .gml files and it will playback in the browser. People have developed their own playback mechanisms, which are some of the first open source collaborations that happened around GML files. There is a user account and you can upload files; people have made image renderers, there are people that have made Flash players, .svg players. Golan Levin has developed an application that converts a .gml file into an auto-CAD format. The 000000book site is basically where graffiti writers connect to developers. In the middle between Graffiti Analysis and database is the Graffiti Markup Language, that I think will have it's own place on the web. But sometimes I see it as one project. One of my interests is in archiving graffiti and all of these things are ways of doing that. It is interesting how these three things work together. In terms of an OS development model it has been producing results I haven't seen when I just released source code.

How do you do that, develop a standard for graffiti?

We started by looking at Graffiti Analysis and L.A.S.E.R. Tag, the apps that were using graffiti motion data. From those two projects I had a lot of experience of meeting graffiti-writers as a userbase. When you meet with them, they tell you right away what pieces of the software they think are missing. So from talking with them we developed a lot of features that now are in GML like brushes, drips, line-thickness. Some people had single line tags and some people had multi-line tags so that issue came up because GML tracks both drawing and non-drawing motion so we knew that we needed in the file format to talk about pen up and pen down. I was interested in the connection points between lines also. We tried to keep it very stripped down. From the beginning we knew that people that would participate as developers or anonymous contributors were not going to be the same people that would develop a Linux core. They are students, people just getting into programming or visual programming. We wanted people to be able to double-click a .gml file and than everything should verbally make sense so it is Begin stroke. End stroke. Anyone with basic programming skills should be able to figure out what's going on.

Did you have any moment where you had to decide: this does not belong to graffiti or: _this might be more for calligraphy tracking_?

The only thing that has to be in there is the format in x-y time scenario with some information on drawing and not drawing, everything else is bonus. So if you load in an XML-file structured like that, compliant apps will load it in. On top of that, there are features that some apps will want and others not. Keywords are, for example, a functionality that we are still developing applications for. It is there but we are looking for how to use it.

Did you ever think about this standard as a way to define a discipline?

(laughs) I think in the beginning it was a very functional conversation. We were having apps running this data and I don't think we were thinking of defining graffiti when we were writing the format. But looking back, it is interesting to think about it. Graffiti has a lot of privacy issues related to it too, right? So we did discuss about what it would mean to start recording geo-located data. There are different interests in graffiti. There is an interest in visuals and in deconstructing characters. Another group is interested in it, because it is a sport and more of a performance art. For this type of interest, it is more important to know exactly where and when it happened because it is different on a rooftop in New York to a studio in the basement of someones house. But if someone realizes this data resulted from an illegal action, and wanted to tie it back to someone, than it starts to be like a surveillance camera. What happens when someone is caught with a laptop with all this data?

Your desire to archive, is it also about producing new work?

I see graffiti-writers as hackers. They use the city in the same way as hackers are using computer systems. They are finding ways of using a system to make it do things that it wasn't intended to do. I am not sure graffiti-writers see it this way, but I am in this position where I have friends that are hackers, playing around with digital structures on-line. Other friends are into graffiti-writing and to me those two camps are doing the most interesting things right now, but these are two communities that hardly overlap. One of the interests I have is making these two groups of people hang out more. I was physically the person bridging these two groups; I was the nerd person meeting the graffiti writers talking to them about software and having this database. Now it is not about my personal collection anymore, it is making a handshake between two communities; making them run off with each other and having fun as opposed to me having to be there all the time to make introductions.

Is GML about the distribution of signature? I mean: The gestures of a specific person can now be reproduced by a larger community. How does that work?

This is an interesting conversation we should have with the graffiti-writers. A tag might be something they have been writing for more than 25 years and that will be very personal to them and the way they write this is because they've written it a million times. So at the one hand it is super-personal, but on the other hand a lot of graffiti writers have no problem sharing this data. To them it is just another tag. They feel like, _I have written this tag a billion times_ and so when you want to keep one of them, it is no big deal. I don't think the conversation has gotten as involved as it could have. You set something in motion and cross your fingers hoping that everyone plays nice and things go well and so far that is what has been happening. But you are dealing with people that are uploading something that is super personal to them and I'd be curious to see what happens in the future. The graffiti taxonomy project that I have been doing involves a lot of photos of graffiti. It is a visual studies based on characters, I am shooting thousands of photos of graffiti and I don't have an opportunity to meet with all these writers to ask them if it is OK. So I get e-mails from writers once in a while saying _Hey, you used a photograph of one of my tags_ and usually it is them feeling out where my intentions are and where I am coming from. It has taken a long time to gain the trust of the community I am working with. Usually when I am able to explain what I am doing and that everything is released openly and meant to be completely free, so far at least the people I have managed to talk toare OK with it and understand it. Initially when people see something they've made being used by other people, a lot of times it can be a point where a red flag is raised and I am assuming there are more red flags going to go up.

If you upload a GML-file, can you insert a license?

Not yet. Right now there is not even a _private mode_ on the 000000book site. If you upload, everything is public. There is a lot of interesting issues with respect to the license that I have been reluctant to deal with yet. Once you start talking too much about it, you will scare off people on either side of the fence. I think that will have to happen at some point but for now I have decided to refer to it as an _open database_ and I hope that people will play nicely, like I said.

But just imagine, what kind of license would you need?

It might make more sense to go for a media-related license than for a code license. Creative Commons licenses would lend themselves easily for this. People could choose non-commercial or pure public domain. Does that make sense?

Well, yes but if you look at the objects that people share, we're much closer to code than to a video-file?

Functionally it is code. But would a graffiti-writer know what GPL is?

I am interested in the apprentice-system you were talking about earlier. Like a young writer learning from someone else they admire. The GML-notation of x-y-time might help someone to learn as well. But would you ever really copy someone else's tag?

One of the reasons I think graffiti-writing has this history of apprenticeship is because you don't really have a chance to learn otherwise. You don't turn on the TV and see someone else doing it. You only see how it is being written if you see other people actually do it. That was one of the original reasons I started doing graffiti-research because, having met with graffiti-writers. I thought: it is a dance, it is as much about motion as it is about how the final image is constructed. You can come to a much better understanding about how it is made as opposed to just seeing a photograph of it.

If you want to learn from the person writing, you would need to see more than just the trace of a pen?

Someones tag might look completely different if they had six seconds to make it, they make different decisions. In the first version of the Graffiti-Analysis project, I had one camera recorder tracking the pen and another camera behind the hand and another so you could see the full body. But there was something about tracking just the pen tip that I liked. It is an easier point of entry for dealing with the motion data than having three different video feeds.

Maybe it is more about meta-data? Not a question of device or application, but about space for a comment.

Maybe in the keywords there will be something like: Rooftop. Brooklyn. Arrested. The most interesting part is often the stories that people tell afterward anyway. So it is an interesting idea, how to tie the story to the data. It is a design problem too. Historically graffiti has been documented many times by outsiders. The movie Style Wars[10] [13]Style Wars. Tony Silver, 1983. Link documentary that was made by outsiders that became insiders. Also, the people that have been documenting most of the graffiti are not necessarily graffiti writers. Graffiti has a history with documentarians entering into their community and playing a role but sharing the stories is something writers do internally, not as much to outsiders. How do you figure out a way to get graffiti-writers to document their stories into the .gml-files themselves, or is it going to take outsiders? How does the format facilitate that?

Do you think the availability of a project like GML can have an impact on the way graffiti is learned? If data becomes available in a community that operates traditionally through apprenticeships and person-to-person sharing, what does it do?

I am interested in open source culture being influenced by graffiti, and I am interested in open source culture influencing graffiti as well. On a big picture I would love it if the graffiti community got interested in these ideas and had more of a skill-sharing-knowledge-base. KATSU[11] [14]KATSU Link, someone I worked with in New York, has acquireda lot of knowledge about how to make tools for graffiti and he initially wasn't so much into sharing them, because graffiti-writers tend to save that knowledge for themselves so that their tags are always bigger and better (laughs). Talking to him I think I convinced him to write tutorials on how to make some of these tools. On the street art side there is Mark Jenkins[12] [15]Mark Jenkins tapesculptures Link, he has this technique of making 3D objects that exist within the city and we had a lot of conversations too. There are many ways tech circles and open source circles can come together with people that are making things outside, with their hands. I think graffiti can learn from that. In the end people would be making more things outside which would be a good thing.

In a way typography has a similar culture of apprenticeship. Some people enjoy spreading knowledge, and others resist in the name of quality control.

Interesting. I think the work I am doing is such a tangent! In general, for something that is decidedly against the rules, the culture of writing graffiti often has a rigid structure. To people in that community what I do is a blip on their radar. I am honored when I get to meet graffiti-writers and they are interested in what I am doing but I don't think it will change anything in what is in some ways a very strict system. And I don't want that either. I like the fact that they found a way to make spray-paint and markers change the way each city in the world looks. They have the tools they need. Digital projectors will not change that. Graffiti-writers still like to see their names projected at big scales in new ways but it is not something they really need (laughs).

And the other way around? How does graffiti have an influence on Open Source communities?

For the people on the technology side, it is an easy jump. To think about hacking software systems and than about making things outside. I see that with the Free Art and Technology Group[13] [16]The Free Art and Technology (F.A.T.) Lab is an organization dedicated to enriching the public domain through the research and development of creative technologies and media. Release early, often and with rap music. Link projects in the city, it takes little to come up with great ideas. I also see that in the class I teach, Urban Hacking. There is already a natural overlap.

What connects the two?

It is really about the idea of hacking. The first assignment in the class is not to make anything, but simply to identify systems in the city. What are elements that repeat. Trying to find which ones you can slip into. It has been happening in graffiti forever. Graffiti in New York in the 80's was to me a hack, a way to have giant paintings circulating in the city... There is a lot of room to explore there.

Your experience with the Blender community[14] [17]Blender is a free open source 3D content creation suite. Link did not sound like an easy bridge?

Recently I released a piece of software that translates a .gml file and translates it into a .stl file, which is a common 3D format. So you can basically take a graffiti gesture and import it into software like Blender. I used Blender because I wanted to highlight this tool, because I want these communities to talk to each other. So I was taking a tag that was created in the streets of Vienna and pulling it into Blender and in the end I was exporting it to something that could be 3D printed, to become something physical. The video that I posted intentionally showed on-line showed screenshots from Blender and it ended up on one of the bigger community sites. I only saw it when my cousin, who is a big Blender user, e-mailed me the thread. There is about a hundred dedicated Blender users discussing the legitimacy of graffiti in art and how their tools are used[15] [18]Link ; pretty interesting but also pretty conservative.

Why do you think the Blender community responded in that way?

It doesn't surprise me that much. Graffiti is hard to accept, especially when we are talking about tags. So the only reason we might be slightly surprised by hearing people in the Open Source community react that way, is because intellectual property doesn't translate always to physical property. Writing your name on someone's door is something people universally don't like. I understand. For me the connection makes sense but just because you make open source doesn't mean you'll be interested in graffiti or street art or vice versa. I think if I went to a Blender conference and gave a talk where I explained sort of where I see these things overlap, I could make a better case than the three minute video they reacted to.

What about Gesture Markup Language instead of [19]Graffiti Markup Language?

Essentially GML records x-y-time data. If you talk about what it functionally does, it is probably more related to gesture than it is to graffiti. There is nothing at the core specifically related to graffiti. I am interested in branding it in relation to graffiti and to get people to talk about open source where it is traditionally not talked about. To me that is interesting. It is a way to get people excited about open data, and popularizing ideas about open source.

Would you be OK if it would get more popular in non-graffiti circles?

I am super excited when I see it used in bizarre places. I'll keep using it for graffiti, but someone e-mailed me that they were upset that it only tracks one point. There hasn't been a need to track multiple tags at once. They wanted to use it to track juggling, but how to track multiple balls in the air? I keep calling it [20]Graffiti Markup Language because I think it is a good story.

What's the license on GML?

We haven't really entered into that. Why would you need a license on a file-format?

It would prevent that anyone could own the standard.

That sounds good. Actually it would be interesting for the project, if someone would try to license it. Legal things matter, but for the things I do, I am most of all interested in getting the idea across.

I am interested in the way GML stems from a specific practice. How it is different and similar to large, legal, commercial, global standardization practices. Related, how can GML connect to other standard practices? Could it be RDF-compliant?

Gesture recognition to help out the police?

Or maps of places that are in need of some graffiti? How to link .gml to other types of data?

It is hard for me to imagine something. But one thing is interesting for example, how GML is used in the EyeWriter project. It has not so much to do with gesture, but more with how you would draft in a computer. TEMPT is plotting points, so the time data might not be so interesting but because it is in the same format, the community might pick it up and do something with it. All the TEMPT data he writes with his eyes and it is uploaded to the 000000book site automatically. That allowed another artist called Benjamin Gaulon[16] [21]Benjamin Gaulon, Print Ball Link who I now know, but didn't know at the time, to use it with his Print Ball project. He took the tag data from a paralyzed graffiti writer in Los Angeles and painted it on a wall in Dublin. Eye-movement translated into a paint-ball gun ... that is the kind of collaboration that I hope .gml can be the middle-point for. If that happens, things can start to extrapolate on either end.

You talked about posting a wish-list and being surprised that your wishes were fulfilled within weeks. Why do you think that a project like EyeWriter, even if it interests a lot of people, has a hard time gathering collaborators, while something much more general like GML seems to be more compelling for people to contribute to?

I'll answer that in a second, but you reminded me of something else: because EyeWriter was .gml based, a lot of the collaborations that happened with people outside of the project were .gml related, not EyeWriter related. So we did have artists like Ben and Golan take data drawn by TEMPT and do completely different things which made TEMPT a collaborator with them in a way. The software allowed him to share his work in a format that allowed other people to work with him. The wish-list came out of the fact that I was working on a graffiti-related project that had a lot of use but not a lot of innovation. Not so many people were using it in ways I wasn't expecting, which is something you always hope of course. By saying: _Here's the things I really would like to happen_, things started to happen. I have been surprised how that drove momentum. Something similar I hope will happen to the work we will do together in the next months too!

What are you planning to do?

We are planning to make a dedicated community page for the graffiti markup language which is one of the three points of the triangle. The second step would be a new addition to the wish-list, a challenge with a prize associated to it which seems funny. The project I'd like to concentrate on is making the data collection easier so that graffiti writers can be more active in the upload sense. Taking the NASA development model: Can you get into orbit on this budget?

How is that different from the way you record graffiti-motion at the moment?

If I go out with a graffiti-writer, I'm stuck standing with a laptop and a camera facing the wall and then the graffiti writer needs to have a really bright light attached to the writing device which is a bit counter-intuitive when you are trying to do something without being seen (laughs). It could be infrared by the way, that could be the first step but then security cameras would still pick it up. The design I am focusing momentum on is a system that's easier. A system that can work without me there, without having to have a laptop there. The whole idea is that it would be a natural way to get good data, to document graffiti without a red-head holding a laptop following you around the whole time!

Paris, December 2010

How is it to be the sole jury member?

I tried to get another jury-member on there actually. Do you know Limor Fried? She runs Adafruit Industries[17] [22]Limor Fried, Adafruit Industries Link She works with her partner Phil Torrone who runs Make Blog[18] [23]Phillip Torrone, Makezine http://makezine.com/pub/au/Phillip_Torrone}. I invited her to be the second jury-member because she makes open source hardware kits; this is her full-time thing. She is very smart and has a lot of background in making DIY kits that people actually build. She is also very straightforward and very busy, so she wrote back and said: this is too much work. No. So... yeah, I am the only jury member. Hmmm.

Is the contest already over?

It is not over. It was easy to launch; I tried to coincide it with the launch of the website and there were a couple of things going on at the same time. The launch helped spread the word about this file-format, and people making projects, and vice versa.

Did you have any proposals that came close to meeting the challenge? Did you consider giving out the prize?

No. There are a couple of people that got really close. The interesting thing that is happening with the challenge is something that is also happening to other high barrier projects: You end up speaking to the people you already work with the most. I have a hard time figuring out to some extent what is really happening, but the things I hear, of people making progress, is people that are close to me. It reminds me of the EyeWriter project where people that are to dip their toes into this, are already in the friend group, or one level removed. They are pretty high level programmers. I didn't really think that actual money would be such an incentive but more that it would make the challenge feel serious, more in the sense of an organization that has some kind of club behind it. If you solved one of the design problems by the Mozilla community you could receive kudo's from the community, but if you solved one of my projects, you don't really get kudo's from my community, do you? Having the money associated makes it this big thing. At Ars Electronica and so on, it got people talking about it and so it is out there. That part worked. Beyond that it has been a bit hard to keep the momentum. Friends and colleagues send me ideas and ask me to look at things, but people I don't know are hard to follow; I don't think they are publishing their progress. There is a hackerspace in Porto that has been working on it, so I see on their blog and Twitter that they are having meetings about this and are working on it.

Don't you think having only one prize produces a kind of exclusivity? It seems logical {} to publish your notes?

Maybe. Kyle[19] [24]Kyle McDonald Link has been thinking up ways to do it and I know he wanted to use an optical mouse, and then this a friend Michael [20] [25]Michael Auger Link has been using sensors, and he ran into a software problem but had the hardware problem more or less solved. And then Kyle, a software expert, has been running into hardware problems and so I kind of introduced them to each other over e-mail so I don't know if they are working on it together.

Would you consider splitting the prize?

I don't care, but I don't know if the candidates would consider splitting the prize! I know Michael has already spent a lot of money because he has been buying Arduino's and other hardware. He wants to make a cheap version to solve the problem and then make another one that costs 150 euro on top of the price limitation to make it easier to use. He is spending a bunch of money so even if he wins, it is going to get him only out of the hole and he will not have much left. Actually, Golan[21] [26]Golan Levin Link had an idea for an iPhone app that he wants to make but I am not sure it solves it.

Why don't you think his app will solve it?

He is really interested in making something where you do not need to meet with the graffiti-writer. His idea was that if you could take a photo of it on the wall, and then with your finger you guide it for how it was written. It has an algorithm for image processing and that combined with your best guess of how it was written would be backed out in motion data. But it is faked data.

That it is really interesting!

Yes it is and I would love it if he would make it but I am not going to let him win with it (laughs). I understand why he wants to do it; especially if you are not inside the graffiti community, your only experience is what you see on the wall and you don't know who these people are and it is going to be almost impossible to ever get data for those tags. If you don't have access to that community you are never going to get the tag of the person that you really want. I like the idea that he is thinking about getting some data from the wall as opposed to getting it from the hand.

Learning by copying. Nowhere near solving the challenge, but interesting. At OSP[22] [27]OSP (Open Source Publishing) is a graphic design collective that uses only Free, Libre and Open Source Software. Link designers are invited into Open Source Software by way of contest. Troy James Sobotka[23] [28]_The very notion of Libre / Free software holds cooperation and community with such high regard you would think that we would be visionary leaders regarding the means and methods we use to collaborate. We are not. We seem to suffer from a collision of unity with diversity. How can we more greatly create a world of legitimate discussion regarding art, design, aesthetic, music, and other such diverse fields when we are so stuck on how much more consistent a damn panel looks with tripe 22 pixel icons of a given flavour?_ Link angry and wrote: _We want to be part of this community, we don't want to compete for it_.

With the EyeWriter project, we were thinking a lot about that; how to spur development. I think I would not have done a competition with the EyeWriter. Making it fun, that is what makes it happen. If it would be a really serious amount of money, with people scraping at each other, fighting each other ... For me, the fact that there is prize money makes something that is already ridiculous in itself even more funny. To have prize money for such a small community of people that are interested in coding {} in graffiti. I'm not seriously thinking that we can spur development with this kind of money. To use the EyeWriter as an example, we've had money infusions from awards mostly and we had to think about how we could use that money to get from point A to point B. That's also a project where we had very definable design goals of what we wanted to reach, especially between the first version and where we are now with the second version.

How did that work?

We are not talking about a ton of money here, 10 to 20.000 euro, and we tried to get as far as we could. We got almost no work done between the meetings in LA but if we flew in, it was OK to take a week out of our schedules and really hammer at it. We were trying to think how we could do the same thing for people that we wanted to work with and who we had met in conferences. So that is how we thought of spending that money. The other way we use money in the EyeWriter project is that we buy people kits. We know a few people that are interested in hacking on it but they don't have the hardware. Not that they are so expensive, but Zach wants to buy twenty or thirty unpackaged kits and he has interns working with him in New York helping to build them. So we have these systems ready so as soon as someone wants to get hacking on it, we can mail them a working system that they can just plug in and they don't have to waste their time ordering all these parts from all these websites all over China. And when they are done, they just send it back.

You talked about some things in the challenge that worked and some that didn't.

I think the forum is the obvious thing that did not work. I have friends working on Open Frameworks, it is headed primarily by Zach and Theo. When you see that forum, it is very involved. It is a deep system, with many different libraries and lots of code flying around. GML is really not large enough. I think what makes sense for this project is when I post news about the project, I see it ripple in Google Alerts. For people working on it, having a place where these things show up is already a lot. The biggest success is the project space, to see all the projects happening.

What happened on the site since we talked?

A project I like, is kml2GML[24] [29]Yamaguchi Takahiro Link by a friend from Tokyo. He was gathering GPS data riding his bike around various cities, and building up a font based on his path. I like projects like this, where someone takes a work that is already done and just writes an application to convert the data into another format. To see him riding his bike played back in GML was really nice. It is super low barrier to entry, he already did all the hard work. I like that there is now a system for piping very different kinds of data through GML.

But it could also work the other way around?

Yeah. This is maybe a tangent but depending on how someone solves the GML-challenge... I was discussing this with Mike (the person that is developing the sensor based version). He was thinking that if you would turn on his system, and leave it on for a whole night of graffiti writing, you would have the gestural data plus the gps data. You could make a GML-file that is tracking you down the street, and zoom in when you start making the tag. Also you would get much more information on 3D movement, like tilt and when the pen is picking up and going down. Right now all I am getting is a 2D view through video data. I am really keeping my fingers crossed. But he ran into trouble though.

Like what?

I have my doubts about using these kind of sensors, because drift is a problem. When you start using these sensors too long, it tends to move a little bit. I think he is working within a 0.25 inch margin of error right now, which is right on the edge. If you are recording someone doing a big piece, this is not going to ruin my day too much but if you record a little tag than it is a problem. The other problem is that you need to orient the system before you start tagging. It needs to know what is up and down, you have to define your plane of access. I don't really understand this 100% but he thinks he can still fit it all within the ten second calibration requirement, he's thinking that each time you come to a wall, you tap once, you tap twice and tap a third time to define what plane you are writing on and that calibrates the 3D space. Once you have that calibration done, you can start writing. It is not as easy as attaching a motion sensor. The problem is hard.

So you need to touch the wall before writing on it, feeling out the playing field before starting! It is like working on a tablet; to move from actual movement to instruction; navigation blends into the action of drawing itself.

I like that!

The guy using the iPhone did not use it as a sensor at all?

Theo was interested in using the iPhone to record motion data in GML, but also to save the coordinates so you could try it into a Google Earth or something but he had trouble with the sensitivity of the sensor. Maybe it is better now but you needed to draw on a huge scale for one letter. You could not record anything small.

But it could be nice if you could record with a device that is less conspicuous.

I know. I have just been experimenting with mounting cameras on spray-cans. A tangent to GML, but related. It is not data, but video.

What do you think is the difference between recording video, and recording data? You mentioned that you wanted to move away from documentation the image to capture movement. Video is somehow indirect data?

Video is annoying in that it is computationally expensive. In Brazil[25] <span class="NOTE">[30] Graffiti Analysis: Belo Horizonte, Brazil 2010 Link not very precise. Kyle thinks he might be able to back out GML data from videos. This might solve the challenge, depending on how many cameras you need and how expensive they are. But so far I have not heard back from him. He said it needs three different cameras all looking at the wall. I mean: talk about computationally expensive! He likes video-processing, he knows some open source software that can look for similar things and knows how to relate them. To me it seems more difficult than it needs to be (laughs).

It is both overcomplicated and beautiful, trying to reverse engineer movement from the image.

I am getting more into video myself. I get more enjoyment from capturing the data than from the projections, like what most people associate with my work.

Why is it so much more interesting to capture, rather than to project?

In part because it stays new, I've been doing those projections for a while now and I know what happens at these events. For a while it was very new, we just did it with friends, to project on the Brooklyn bridge for example. Now it has turned into these events where everyone knows in advance, instead of just showing up at at a certain time ate a set corner. It has lost a lot of its magic and power. Michele and I have done so many of these projections and we sort of know what to expect from it, what questions people will ask. When I meet with graffiti writers, that almost always feels new to me. When we went to Brazil, we intentionally tried to not project anything but to spend as much time as possible with writers. Going out with graffiti writers to me always feels right.

Is the documentation an excuse to be taken along, or is the act of documenting itself interesting to you?

To me documentation is interesting. I don't know where all of this is going right now, I am just trying to get the footage; I put these pieces together showing all this movement but I don't really know what the final project is. It is more about collecting data so I am interested in having video, audio and GML that can be synced up, and the sound from these microphones is something to do something with later. This is research for me. I like the idea of having all this data related to a 10 second gesture. I am thinking that in the future we can do interesting things with it. I am even thinking about how the audio could be used as a signal to tell you what is drawing and what is not drawing. It is a really analog way of doing it, but in that way you don't need a button where you are getting true and false statements for what is drawing and what is not drawing; you can just tell by the sound: tfffpt ... tfffpt.

You can hear the space, and also the surface.

I got started doing this because I love graffiti and this is a way to get closer to it again. Like getting back out to the streets and having very personal relationships to the graffiti-writers and talking to them, and having them give feedback. I think that is how the whole challenge started. It didn't start because I was projecting, but because I was out on the street and testing the capture, having graffiti-writers nearby when it is happening. It feels like things are progressing that way.

Are you thinking of other ways of capturing? You talk about capturing movement, but do you also archive other elements? Do you take notes, pictures? What happens to the conversations you are having?

I have been missing out on that piece. It is a small amount of time we have, and I am already trying to get so much. I am setting up a camera that shoots straight video from a tripod, I am capturing from the laptop and I am also screen-casting the application, my head is spinning. One reason I screwed up this footage in the beginning is because with all these things going on I forget to turn on some things. Maybe someone will solve this challenge.

Are you actually an embedded anthropologist?

In the back of my head I am thinking this will become a longer documentary. I like to experiment with documentation, whether that is in code or with video. I do think that there is this interesting connection between documentation and graffiti and how these two things overlap. I am always thinking about documentation. The graffiti-writer that was in Vienna[26] [31]momo3010 Link showed me a video that was amazing. It was him and a friend going out on a sunny day at 15:30 in the afternoon with two head mounted cameras, bombing an entire train and you hear the birds singing and you only experience it by these two videos that are linked. There are interesting constraints: your hands are already full, you don't want peoples' faces on camera so the head-mounted cameras were smart. Unless you walk in front of a mirror (laughs).

Is it related to the dream of self documenting code?

I like that. Even doing the challenge is in a way a reflection on this, how I am fighting to get GML back to the streets somehow, it has a natural tendency to get closer to the browser, to the screen, and my job is to get it back to the street. It is so sexy and fun and flashy and that is important too. My job is to keep the graffiti influence on it as large as the other part.

Is any of this reflected in the standard itself?

I haven't looked at the standard for a while now.

I was thinking again about live coding and notation. Simon Yuill[27] [32]Simon Yuill. All problems of notation will be solved by the masses. Mute Magazine, 2008} describes notation as a shared space that allows collaboration but also defines the end of a collaboration.

Maybe using an XML-like structure was a bad idea? Maybe if I had started with a less code-based set of rules? If the files were raw video, it would encourage people to go outside more often? By picking XML I am defining where the thing heads in a way. I think I am OK in the role of fighting that tendency. It is not just a problem in GML but with a lot of work I have been doing with graffiti and technology and even way back with Graffiti Analysis, before GRL (Graffiti Research Lab), the idea was always to keep the research very close to the people doing graffiti. I was intentionally working with people bombing a lot and not with graffiti celebrities. I wanted to work with who's tag was on my mailbox, who's tag do I see a million times when I walk down the street. Since then a lot has happened, like with more popular projects such as L.A.S.E.R. Tag, and it goes almost always further away from graffiti. Maybe that is a function of technology. Technology, or the way it is now, will always drift towards entertainment uses, commercial uses.

Do you think a standard can be subversive? You chose XML because it is accessible to amateur programmers. But it is also a very formal standard, and so the interface between graffiti-writers and hackers is written in the language of bureaucracy.

(laughs) I thought that there was something funny with that. People that know XML and the web, they get the joke that something so rigid and standardized is connected to writing your name on the wall. But to be honest, it was really just a pragmatic choice.

It reminds me of an interview[28] [33]Interview with Francois Chastanet Link Chastanet who wrote a book[29] [34]Francois Chastanet, Cholo writing: Latino gang graffiti in Los Angeles. Dokument, 2009} about tagging in Los Angeles. He explains that the Gothic lettering is inspired by administrative papers!

I am wondering whether you're thinking about the standard itself as a space for hacking?

Graffiti is somehow coded in-itself. Do you mean it would be interesting to think how GML could be coded in a way for graffiti-writers, not for coders? There would be more space for that when more people start to program at a younger age? When it is more common knowledge. If I would start to do that now, I would quickly lose my small user-base. I love that idea though; the way XML is programmed fits very much to the way you program for the web. But what if it was playing more with language, starting from graffiti which is very coded?

When I was in college, I was always thinking about how to visualize motion in print. I was looking for ways people had developed languages for different ways of writing.

Maybe you could look at the Chinese methods for teaching writing, because the order of the strokes is really important. If you make the stroke from bottom to top, and not from top to bottom, it is wrong.

A friend in Hong Kong, MC Yan, loves the Graffiti Analysis project because it shows the order in which he is writing and he likes to play with that. So he writes words in different order than people are used to and so it changes the meaning. People can not only watch the final result, but also the order which is an interesting part of the writing process. The brush, the angle, direction: depicting motion! In the beginning of the Graffiti Analysis Research project I was very against projection, because I felt that was totally against the idea of graffiti. I was presenting all of these print ideas and the output would be pasted back into the city because I was against making an impermanent representation of the data. In the end Zach said, you are just fighting this because you have a motion project and you want to project motion and then I said alright, I'll do a test. And the tests were so exciting that I felt OK with it.

In what way does GML bridge the gap between digital drawing and hand writing? Could you see a sort of computer-aided graffiti? Could you see computation enter graffiti?

Yeah. When you are in a controlled environment, in a studio, it is easy but the outdoors part always trips me up. That is why the design constraints get interesting, playing in real time with what someone is writing. I think graffiti-writers would be into that too. How to develop a style that is unique enough to stand out in an existing canon is already hard enough. This could give someone an edge.

I think the next challenge I'd like to run is about recreating the data outside. I've been thinking about these helicopters with embedded wireless camera's, have you seen them? The obvious thing to me would be uploading a GML-file to one of these helicopters that is dripping paint on a rooftop. Scale is so important, so going bigger is always going to be better. Gigantic rooftop tags could be a way to tie it back to the city, give it a reason? I am thinking of ways to get an edge back to the project. The GML-challenge is already a step into that direction; it is not about the prettiest screensaver. To ask people to design something that is tying back to what graffiti is, which is in a way a crime. I think fixing the data capture is the right place to start, the next one could be about making marks in the city. Like: the first person to recreate this GML-tag on the roof of this building, that would be fun. The first person that could put this _Hello World_ tag onto the Brooklyn bridge and get a photo of it gets the prize. That would get us back to the question of how we leave marks on the surface of the city.

When you capture data of an individual writer in a certain standard, it ends up as typography?

That's another trend that happens when designers look at graffiti, and I've fallen into this too sometimes, you want to be able to make fonts out of it. People have done this actually; there's a project in New York where they met with pretty influential graffiti writers and asked them to write in boxes, the whole alphabet, and I think there's something interesting there. The alphabet that you saw the robot write was drawn by TEMPT with the EyeWriter and what he did was a little bit smarter than other attempts by graffiti writers to make fonts. He intentionally picked a specific style, the Cholo style, and the format is very tall, vertically oriented, angled. That style is less about letter connections and pen-flow. What graffiti has developed into, and especially tags, is very much about how it is written and the order of the letters. When TEMPT picked this style he made a smart decision that a lot of people miss when you make a font, you miss all the motions and the connections.

What if a programmer could put this data in a font, and generate alternating connections?

That kind of stuff is interesting. It would help graffiti-writers to design tags maybe? To get my feet wet, I designed a tag once, and it was so not-fun to write! I was thinking about a tag that would look different and that would fit into corners, I was interested in designing something that wasn't curved; that would fit the angles of the city, hard edges. So I had forgotten all my research about drafting and writing. I think I stopped writing in part because the tag I picked wasn't fun o write. For a font to work like writing, it is not just about possible connections between lines. You'd need another level in the algorithm, the way the hand likes to move.

It would be a good algorithm to dream up. It was beautiful to see a robot write TEMPT's letters by the way.

When TEMPT saw the robot writing for the first time, his reaction was all about the order of how the letters were constructed. The order is I think defined by the way he dropped the points in with the EyeWriter software. When he was writing with his eyes, he ended up writing in the same way as he would have written with his hands. When he saw the video with the robot, it freaked him out because he was like: _That's how my hand moved when I did that tag!_

The [35]Graffiti Markup Field Recorder challenge

An easily reproducible DIY device that can unobtrusively record graffiti motion data during a graffiti writer's normal practice in the city.[30] [36]GML-recorder challenge as published on: Link

Project Description and Design Requirements:

The GML Field Recorder Challenge is a DIY hardware and software solution for unobtrusively recording graffiti motion data during a graffiti writer's normal practice in the city. The winning project will be an easy to follow instruction set that can be reproduced by graffiti writers and amateur technologists. The goal is to create a device that will document a night of graffiti bombing into an easily retrievable series of [37]Graffiti Markup Language (.gml) files while not interfering with the normal process of writing graffiti. The solution should be easy to produce, lightweight, cheap, secure, and require little to no setup and calibration. The winning design solution will include the following requirements listed below:

Material costs for the field device must not exceed 300 euros.

300 euros even felt expensive to me. How can this be a tool that is really accessible? If it goes over a certain price point, it is not the kind of thing that people can afford to make. It is a very small community, a lot of the people that are going to have enough interest to build this are not going to have a background in engineering, and are probably not even a part of the maker scene that we know. The audience here might not be people that are hanging out on Instructables. I wanted to make sure that the price point meant that people could comfortably take a gamble to make something for the first time. But I also did not want to make it so small that the design would be impossible.

Computers and equipment outside of the 300 euros can be used for non-field activities (such as downloading and manipulating data captured in-field), but at the time of capture a graffiti writer should have no more than 300 euros worth of equipment on him or herself.

I was trying to think of how the challenge could be gamed... I did not want to get into a situation where we were getting stressed out because some smart hacker found a hole in the brief, and bought a next generation iPhone that somehow just worked. I didn't want to force people to buy expensive equipment. This line was more about covering our own ass.

The graffiti writer must be able to activate the recording function alone (i.e., without assistance from anyone else).

Are you going to be out of work soon?

Thinking selfishly, I screw up on documentation a lot because I have too many hats. When I'm going out doing this, I am carrying a laptop, a calibration set up, I also have one video-camera on me that is just documenting, I have another one on a tripod, and I am usually screen capturing the software as it processes the video-footage because it tells another story. I screw up because I forget to hit stop or record. If the data-capture just works, I can go have fun getting good video-footage.

What if it had to be operated by more than one person? It is nice how the documentation now turns the act of writing into a performance-for-one.

If you record alone, the data becomes more interesting and mysterious, right? I mean, no one else has seen it. Something captured very privately, than gets potentially shared publicly and turned into things that are very different. I also thought: you don't want to be dependent on someone else. It is a lot to ask, especially if you are doing something illegal.

Any setup and/or calibration should be limited to 10 seconds or less.

This came out of me dealing with the current system. It feels wrong that it takes ten to fifteen minutes to get it running. Graffiti is not meant to be that way. This speaks to the problem of the documentation infringing on the writing process, which ideally wouldn't happen. The longer the set-up takes, the more it is going to influence the actual writing. It is supposed to be a fly on the wall.

Does it scale? Does a larger piece allow longer callibration -time?

That's true. But I think this challenge is really about recording tags.

All hardware should be able to be easily concealed within a coat with large pockets.

A hack to get around that would have been to design a jacket with ten gallon pockets! I put it there again, to make the device not be intrusive. A big part of graffiti writing is about gaining entry and you limit where you can go depending on how much equipment you have. How bulky it is, what walls you can get up, what holes you can get through.

The winning solution should be discrete and not draw any added attention to the act of graffiti writing.

It's part of the same issue, but this one also came out from me going out and trying to capture with a system where it requires you to attach a flashlight to a graffiti-implement. I didn't want anyone solving the problem and then, Step one is: _Attach a police siren to a spray-paint can_.

The resulting solution should be able to record at least 10 unique .gml tags of approximately 10 seconds each in length in one session without the need for connecting to or using additional equipment.

I wasn't thinking this was going to be an issue in terms of memory-storage, but maybe in terms of memory management. I did not want the graffiti-writer to behave as if he was on vacation with a camera that could take only three photos. I wanted to make sure they were not making decisions on what they were writing based and how much memory they had.

All data recorded using the field recorder should be saved in a secure and non-incriminating fashion.

(laughs) If I had to do that one again, I would have put that in Bonus category actually. That's a difficult question to ask. What does secure mean? It seems a bit unfair, because it doesn't fit in to the way graffiti is currently documented. There's not a lot of graffiti-writers that currently are shooting encrypted photo's and video's, right? But whatever bizarre format comes out from the sensor will help. I don't think that the NYPD will have time or make the effort to parse it. They'd just have a file with a bunch of numbers. Time stamped GPS-coordinates would be more dangerous.

What would count as proof?

In most cases it is hard to convict someone on the basis of a photo of a tag that you would tie to another tag. For good reasons, because if it is a crew name for example, all of a sudden you are pinning one tag on a person that could have been written by twenty people. This came up in a trial in DC when an artist named BORF got arrested. He had written his name everywhere, completely crushed DC and his trial was a big deal. This issue came up and they argued that BORF was a collective, not an individual. Who knows if that's true, there were a lot of people around him, but how do you really know?

GML could help balance the load?

You mean it would not be just the image of a tag but more like signing at the bank?

I mean that if you copy and distribute your data, the chance is small that you can link it to an individual.

The winning design will have some protection in the event that the device falls into the wrong hands.

This again should probably have been a bonus item. Wouldn't it be awesome if you could go home and log in and flip a one to a zero and the evidence goes up in smoke? One graffiti writer friend told me: _If the police comes, just smash the camera as hard as you can!_ It's a silly idea, but it shows that they are thinking about it.

Edible SD cards?

That would be a good idea!

Data should be able to be captured from both spray cans and markers.


Are you prepared for tools that do not exist yet?

That was kind of what I was thinking there. Markers are about direct contact, spray-paint is in free space. If it works in those two situations, you should theoretically be able to tie it to anything, even outside of graffiti. If it was too much about spray-paint, it would be harder for someone to strap it to a skate-board.

System should be able to record writing on various surfaces and materials.

It is something you can easily forget about. When you are developing something in the studio and it works well against a white wall, and than when you go out in the city than you realize that brick is a really weird surface. Or even writing on glass, or on metal or on other reflecting surfaces that could screw up your reading. It is there as a reminder for people that are not thinking about graffiti that much. The street and the studio are so different.

Data should be captured at 30 points per second minimum.

I was assuming that lots of people were going to use cameras, and I wanted to make sure they were taking enough data points. With other capturing methods it is probably not such a problem. Even at 30 points per seconds you can start to see the facets if you zoom in, so anything less is not ideal.

The recording system should not interfere with the writer's movements in anyway (including writing, running and climbing).

So this is where Muharrem is going to run into trouble. His solution interferes. Not that much if you are just working in front of your body space. But the way most writers write is that they are shuffling their feet a lot, moving down the wall. Should it have said: _Graffiti-writer should retain access to feet functionality_? This point should be at the top almost.

To me it feels strange, your emphasis on the tool blending into the background. You could also see Muharrem's solution as an enhancing device, turning the writer into a tapdancer?

I want to have on record: I love his solution! There's a lot in his design that is making us more aware of what's happening in the creation of a tag. One thing that he is doing that is not in the specs, is that he is logging strokes, like up and down. When you watch him using it, you can see a little light going from red to green when the fingers goes on and off the spray-paint can. When you watch graffiti, it is too small of a movement to even notice but when you are seeing that, it adds another level of understanding of how they are writing.


All aspects of the winning design should be able to be reproduced by graffiti writers and amateur technologists.

It wouldn't be exciting if only ten people can make this thing. This tool should not be just for people that can make NASA-qualified soldering connections. Ideally it should not have any soldering. I always thought of a soldering iron like a huge barrier point. I'm all for duct-taped electrical connections.

There's nothing about weather-resistant in the challenge. You're not thinking about rain, are you?

A lot of paint stops working in rain too. I think what you get from this brief though is that the whole impetus for this project is about me trying to steer the ship that clearly wants to go into another direction, back to my interest in what graffiti is rather than anything that people might find aesthetically pleasing. It is not about graffiti influenced visuals.

All software must be released open source. All hardware must include clear DIY instructions / tutorials. All media must be released under an open content license that promotes collaboration (such as a Free Art License or Creative Commons share-alike license).

I didn't want it to be too specific, but there had to be some effort into making it open.

The recording must be an unobtrusive process, allowing the graffiti writer to concentrate solely on the act of writing (not on recording). The act of recording should not interfere with the act of graffiti writing.

I've been through situations where the process gets so confusing that you can't keep your head straight and juggle all the variables. Your eyes and ears are supposed to tell you about who's coming around the corner. Is there traffic coming or a train? There are so many other things you need to pay attention to rather than: _Is this button on?_ The whole project is about getting Good Data. As soon as you force people to think too much about the capture process, I think it influences when and how they are writing. Bonus, but not required:

Inclusion of date, time and location saved in the GML file.

Yes. Security-wise that is questionable, but the nerd in me would just love it. You could get really interesting data about a whole night of writing. You could see a bigger story than just that of a single tag. How long did it take to gain entry? How long were they hiding in the bushes? These things get back to graffiti as a performance art rather than a form of visual art.

Paris, November 2011

Last time we had contact we discussed how to invite Muharrem to Brussels[31] [38]By early October 2011 no winning design-solution had been entered, besides a proposal from Muharem Yildirim that came more than halfway. We decided to use the prize money to fly Muharrem from Phoenix (US) to Brussels (BE) and document his project in a worksession as part of the Verbindingen/Jonctions 13 meetingdays. Link it seems there are new developments?

I think in terms of the actual challenge, the main update is that since we extended the deadline and made another call, I got an e-mail right on the deadline today from Joshua Noble[32] [39]Joshua Noble Link with a very solid and pretty smart proposal that seems to solve (maybe unfortunately for Muharrem) a bit more of the design-spec. It does it for cheaper and does it in a way that I think is going to be easier to make also. His design solution is using an optical mouse and he changed the sensors so it has a stronger LED. He uses a modified lens on top of a plastic lens that comes on top of a mouse, so that it can look at a surface that is a set distance away. It has another sensor that looks at pitch, tilt and orientation, but he is using that only to orient, the actual data gets recorded through the mouse. It can get very high resolution, he is looking at up to a millimeter I guess.

Muharrem's solution seems less precise?

I think he gets away with more because his solution is only for spray-paint and once you are writing on that scale, even if you are off a few centimeters, it might not ruin the data. If you look at the data he is getting, it actually looks very good. I don't think he has any numbers on the actual resolution he is getting but if you were using his system with a pen, I think it would be a different case. I like a lot of his solution too, it is an interesting hack. It is funny that two of the candidates for the prize are both mouse hacks. One is hacking a mechanical mouse and the other an optical mouse.

It goes from drawing on a screen, to drawing on a wall?

And back again!

Yes. When I first was working on graffiti-related software, the whole reason I was building Graffiti Analysis as a capture application was because I did not want to hand graffiti-writers a mouse (laughter). I had done all this research into graffiti and started to be embedded in the community and I knew enough about the community that if you were going to ask them to take part in something that was already weird, you could not give them a mouse and expect any respect on the other end of that conversation. They respect their tools, so the reason I was using camera-input was because I wanted to have a flexible system where they could bring in anything and I could attach a device to it. Now I am coming back to mice finally.

Now the deadline has passed, do you think the passage from wishlist to contest worked out?

I think it was a good experiment, I am not sure how clever it was. To take a piece of culture that a lot of people don't even look at, or look at it and think it is trash, to invest all this time and research and software expertise into it makes people think about the graffiti practice and what it actually is. The cash prize does something similar. It attaches weight to something that most people don't even care about. Even having the name of an organization like Constant attached to it is showing that I am really serious about this. In that sense it is different than a wishlist. I just read the Linus Torvalds[33] [40]Torvalds, Linus; David Diamond (2001). Just For Fun: The Story of an Accidental Revolutionary. New York, New York, United States: HarperCollins.} biography, and I liked his idea that fun is part of innovation, right? In a programming sense, it is scratching a personal itch. The attachment of a prize is more to underline the fun aspect than anything else.

I am still puzzled about GML and how it is at the one hand stimulating collaboration and sharing, and than it comes back to the proud individual that wants to show off. It is kind of funny actually that now two people are winning the prize.

I understand what you mean.

Also in F/LOSS, under the flag of open and free there is a lot of competition. Do you feel that kind of tension in your work?

Even Open and Free are in competition! In a project like White-Glove Tracking for example, the most popular video I had not made and it did not have my name on it but personally I still felt a part of it. I think when you are working in open systems, you take pride when a project has wings. It is maybe even a selfish act. It is the story of me receiving some art-finding and realizing that I am not the best toolmaker for the job. Who ever manages to win the prize gets all the glory, but I'm still going to feel awesome about it.

I have been reading the interview that Kyle McDonald did with Anton Marini[34] [41]Anton Marini: _Some personal projects of mine, for example specific effects and 'looks' that I have a personal attachment to, I don't release_ Link and at some point he talks about being OK with sharing code and libraries, but when it is too much of a personal style, then it is hard to share.

Yes, I thought that was an interesting point. I've been in similar conversations on listservs with artists in the OpenFrameworks, Processing and visual programming communities. What are the open pieces? It makes sense to share libraries, but if I make a print from a piece of code, do I then have to share the exact source and app for how that exact print was made? What does it mean when I am investing money in a print, and it is a limited series but I'm sharing the code? The art world is still based on scarcity and we're interested in computers that are copy-machines. I see both sides of the argument and I am still trying to see how I fit into it. It gets trickier when you are asked to release a piece rather than a tool. If you are an open source artist and you make a toolset, that is easier to share because people use that to make their own things. But then an artist gets asked: how come I can't get the file of that print? I think that is a really hard question.

But isn't the tool often the piece, and vice versa?

I agree. And I haven't solved that question yet. Lately I've been a lot less excited about running workshops for example. A lot of the people that want to take part in the workshops are actually the opposition. Often they own a club and they want to install a cool light-show or they are into viral marketing. I never know which way to go with that. It depends on what side of the curve of frustration I am on at that moment.

Earlier you brought up the contrast between people that were more visually invested and others that are more interested in the performance aspect. I wanted to hear a bit more about the continuum in the culture and how GML fits into that?

My focus has been on tags, this one portion of graffiti. I do think there could be cool uses for more involved pieces. It would be great if someone else would come in and do that, because it is a part of graffiti that I haven't studied that much. I would not even be able to write a specs-sheet for it; it requires a lot of different things when you paint these super-involved murals, when you have an hour or more time on your hands a lot more things come into play. Color, nozzles, nozzle changes and so on.

Z-axis becomes important?

Yes, and your distance from the wall, a lot of other things my brain isn't wrestling with. I think tags are always fundamental, even if they are painting murals that take three days to paint, somewhere in their graffiti education they start with the tags. You're still going to be judged by the community based on how you sign your name on the blackbook. Graffiti is funny because it is almost conservative in terms of how a successful graffiti writer is viewed and it is reflected in how graffiti is in some way similar in the world. In some way it is a let down, to travel from Brooklyn to Paris to Brussels and it looks all the same but I think it stems from the fact that the community is so tight-knit. But at the end of the day it comes back to the tag always. In terms of the performance, in a tag the relationship between form and function is really tight. The way your hand moves and how the tag actually looks on the wall is dictated by the gesture you are making. A piece where you have three hours, that tight synchronization isn't there. With a tag, every letter looks the way it does because that's how it needs to be drawn, because it needs to be connected to this other letter. There's a lot of respect for writers that do oneliners, and even if your tag has more than one line, a good graffiti writer has often a one-line version. If you don't have to pick up the pen it is a really economical stroke.

It is almost like hacking the limitations of gesture.

It is a very specific design requirement. How to write a name that is interesting to think about and to look at, you have to do it in 5 seconds, you have to do it in one line, you have to do it on each type of surface. On top of that, you have to do it a million times, for twenty years.

In Seattle they call a piece that stays up for a longer time a burner. I was connecting that to an archival practice of ephemera. It is a self-agreed upon archival process, and it means that the piece will not be touched, even for years.

Graffiti has an interesting relationship to archiving. On the one hand, many graffiti writers think: Now that tag's done, but I've got another million of them. While others do not want people painting over them, the city or other graffiti writers. Also if a tag has been up there for a few years, it acquires more reverence and it is even worse when it is painted over. But I think that GML is different, it is really more similar to a photo of the tag. It is not trying to be the actual thing.

Once a tag is saved in GML, what can be done with the data?

I am myself reluctant to take any of these tags that I've collected and do anything with it at all without talking closely to whoever's tag it is, because it is such an intimate thing. In that sense it is strange to have an open data repository and to be so reluctant to use it in a way that is looking at anyone too specifically. The sculpture I've been working on is an average from a workshop; sixteen different graffiti writers merged into one. I don't want to take advantage of any one writer. But this has nothing to do with the license, it is totally a different topic. If someone uploads to the 000000book site, legally anyone should be able to do anything that they can do under the Creative Commons license that's on the site but I think socially within the community, it is a huge thing.

There must be some social limits to referentiality. Like beat-jacking for DJ's or biting rhymes for MC's, there must be a moment where you are not just homaging, but stealing a style.

I've seen cases where both parties have been happy, like when [42]Yamaguchi Takahiro used some GML data from KATSU and piped it into Google Maps, so he was showing these big KATSU tags all over the earth which was a nice web-based implementation. I think he was doing what a graffiti writer does naturally: Get out there and make the tag bigger but in different ways. He is not taking KATSU-data from the database without shining light back on him.

GML seems very inspired by the practice of Free Software, but at the same time it reiterates the conventional hierarchies of who are supposed to use what ... in which way ... from who. For me the excitement with open licenses is that you can do things without asking permission. So, usage can develop even if it is not already prescribed by the culture. How would someone like me, pretty far removed from graffiti culture ever know what I am entitled to do?

I have my reasons for which I would and would not use certain pieces of data in certain contexts, but I like the fact that it is open for people that might use it for other things, even if I would not push some of those boundaries myself.

Even when I am sometimes disappointed by the actual closedness of F/LOSS, at least in theory through it's licensing and refusal to limit who is entitled and who's not, it is a liberating force. It seems GML is only half liberating?

I agree. I think the lack of that is related to the data. The looseness of it's license makes it less of an invitation in a sense. If the people that put data up there would sit down and really talk about what this means, when they would really walk through all the implications of what it means to public domain a piece, that would be great. I would love that. Then you could use it without having to worry about all the morality issues and people's feelings. It would be more free. I think it would be good to do a workshop with graffiti writers where beyond capturing data, you reserve an hour after the workshop to talk to everybody about what it would mean to add an open license. I've done workshops with graffiti writers and I talked to everyone: _Look, I am going to upload this tag up to this place where everyone can download them after the workshop, cool?_ And they go cool. But still, even then, do I really feel comfortable that they understand what they've gotten into? Even if someone has chosen a share-alike license, I would be nervous I think. Maybe I am putting too much weight on it. People outside Free Software are already used to attaching Creative Commons licenses to their video's. Maybe I am too close to graffiti. I still hold the tag as primal!

It is interesting to be worried about copyright on something that is illegal, things you can not publicly claim ownership of.

Would you agree that standards are a normalizing practice, that in a way GML is part of a legalizing process?

For that to happen, a larger community would have to get involved. It would need to be Gesture Markup Language, and a community other than graffiti-writers would need to get involved.

Would you be interested in legalizing graffiti?

No. That's why I stopped doing projections.

Not legal forms of graffiti, but more like the vision of KRS One of the Hip-Hop city,[35] [43]Peace and blessings, KRS One Temple of HipHop Link legal. Does that fundamentally change the nature of graffiti?

To me it is just not graffiti anymore. It is just painting. It changes what it is. For me, its power stems from it being illegal. The motion happens because it is illegal.

In a sense, but there is also the calligraphic aspect of it. In Brooklyn, a lot of the building owners say: _yeah, throw it up_ and those are some of the craziest pieces I know of, not from a tag-standpoint, but more as complex graffiti visuals.

I am always for de-criminalization. I don't think anyone should go to jail over a piece of paint that you could cover over in 5 seconds. And that KRS One city you mentioned would be cool to see.

It is his Temple of Hip Hop, the idea to build a city of hip hop where the entire culture can be there without any external repression. It's an utopian ideal obviously.

Of course I would like to see that. If nothing else, you would totally level the playing field between us and the advertisers. The only ones that would get up messages in the city would be the ones with more time on their hands.

At the risk of stretching coherency, hiphop and free software are both global insurgent subcultures that have emerged from being kind of thrown away as fads and then become objects of pondering in multinational boardrooms. So I was hoping to open you up to riff on that: zooming out, GML is a handshake point between these two cultures, but GML is a specific thing within this larger world of FLOSS and graffiti in the larger world of hiphop. What other types of contact points might there be? Do you see any similarities and differences?

For me, even beyond technology and beyond graffiti it all boils down to this idea of the hack that is really a phenomenon that has been going on forever. It's taking this system that has some sort of rigidity and repeating elements and flipping it into doing something else. I see this in hiphop, of course. The whole idea of sampling, the whole idea of turning a playback device into a musical instrument, the idea of touching the record: all of these things are hacks. We could go into a million examples of how graffiti is like hacker culture. In terms of that handshake moment between the two communities, I think that is about realizing that its not about the code and in some sense its not about the spray-paint. There's this empowering idea of individual small actors assuming control over systems that are bigger than themselves. To me, that's the connection point, whether its hiphop or rap or programming. The similarities are there. I think there are huge differences in those communities too. One of them is this idea of the hustler from hiphop: the idea of hustling doesn't have anything to do with the economy of gift-giving. The idea that Jay-Z has popularized in hiphop and that rap music and graffiti have at their core has to do with work ethic, but there's also a kind of braggadocio about making it yourself and attaining value yourself and it definitely comes back to making money in the end. The idea of being 'self-made' in a way is empowering but I think that in the open source movement or the free software movement the idea of hustling does not apply. It's not that people don't hustle on a day to day basis. You disagree with me?

It's interesting because the more you were talking, the more I was not sure of whether you were speaking about hiphop or free software or maybe even more specifically the open source kind of ideological development. You have people like David Hannemeier Hansson who developed Ruby on Rails and basically co-opted an entire programming language to the point where you can't mention Ruby without people thinking of his framework. He's a hustler du jour: this guy's been in Linux Journal in a fold-out spread of him posing with a Lamborghini or something. Talk about braggadocio! You get into certain levels or certain dynamics within the community where its really like pissing contests.

I like that, I think there's something there. At the instigation of the Open Source Initiative, though: like Linus pre-stock option, sitting in his bedroom not seeing the sun for a year and hacking and nerding out. To me they are so different, the idea of making this thing just for fun with a kind of optimistic view on collaboration and sharing. I know it can turn into money, I know it can turn into fame, I know it can turn into Lamborghinis but I feel like where its coming from is different.

I agree, that's clearly a distinction between the two. They are not coming from the same thing. But for me its also interesting to think about it in terms that these are both sort of movements that have at times been given liberational trappings, people have assigned liberatory powers to these movements. Statistically the GPL is considerably more popular than the open source licenses, but I don't know if you sat everybody down and took a poll which side they would land on, whether they were more about making money than they were about sharing. Are people writing blogposts because they really want to share their ideas or because they want to show how much cooler they are?

You're totally right and I think people in this scene are always looking for examples of people making money, succeeding, good things coming to people for reasons that aren't just selflessness. People that are into open source usually love to be able to point to those things, that this isn't some purely altruistic thing.

Maybe you could take some of the hustle and turn it into something in the free software world, mix and match.

I think this line of inquiry is an interesting one that could be the subject of a documentary or something. These communities that seem very different until you start finding things that at their core really really similar.

It would be so interesting to have a cribs moment with some gangsta or rapper who came from that, and he's sort of showing off his stuff and he has this machismo about him. Not necessarily directly mysognistic but a macho kind of character and then take a nerd and have them do the same.

Would they really be so different?

Obviously some rappers and some nerds, I mean that's one of the beauties-{}-I mean its a global movement, you can't help but have diversity-{}-but if we're just speaking in generalizations?

There's a lot of showing off in F/LOSS too.

Yeah, and there's a lot of chauvinism. And when you said that self-made thing, that's the free software idea number one.

I think that part is a direct connection.

And they're coming from two completely different strata, from a class-based analysis which is absent from a lot of discussion. Even on that level, how to integrate them to me is a political question to some degree.


Will any features of GML ever be deprecated?

Breaking currently existing software? I hope not.

Basically I'm asking for your long-term vision?

When the spec was being made of course it wasn't just me, it was a group of people debating these things and of course nobody wants things to break. The idea was that we tried to get in as many things as we could think of and have the base stay kind of what it was with the idea that you could add more stuff into it. It's easy enough to do, of course its not a super-rigid standard. If you look at what the base GML file is, the minimum requirements for GML to compile, its so so stripped down. As long as it just remains time/x-y-z, I don't think that's going to change, no. But I'm also hoping that I'm not gonna be the main GML developer. I'm already not, there's already people doing way more stuff with it than I am.

How does it work when someone proposes a feature?

They just email me (laughs). But right now there hasn't been a ton of that because it's such a simple thing, once you start cramming too much into it it starts feeling wrong. But all its gonna take is for someone to make a new app that needs something else and then there will be a reason to change it but I think the change will always be adding, not removing.