So what should we talk about?

GML stands for [1]Graffiti Markup Language[1] [2]Graffiti Markup Language (.gml) is a universal, XML based, open file format designed to store graffiti motion data (x and y coordinates and time). The format is designed to maximize readability and ease of implementation, even for hobbyist programmers, artists and graffiti writers. Link designed for amateur programmers. It is a way to store graffiti motion data. I started working with graffiti writers, combining graffiti and technology back in New York, in 2003. In graduate school, my thesis was on graffiti analysis, and writing software that could capture their gestures, to archive motion data from graffiti-writers. Back than I was saving the data in an x-y-time array, I was calling them .graph files and I sensed there was something interesting about the data, the visualization of motion data but I had never opened up the project at that time. About a year ago I released the second part of the project, of which the source code was open but the dataset wasn't. In conversation with a friend of mine named Theo[2] [3]Theo Watson Link project[3] [4]In its simplest form, L.A.S.E.R. Tag is a camera and laptop setup, tracking a green laser point across the face of a building and generating graphics based on the laser's position which then get projected back onto the same building with a high power projector. Link .graph file again and how we could bring back the file format as a way to connect all these different applications. Graffiti Analysis[4] [5]Graffiti Analysis is a digital graffiti blackbook designed for documenting more than just ink. http://graffitianalysis.com}, L.A.S.E.R. Tag, EyeWriter[5] [6]The EyeWriter is a low-cast eyetracking system originally designed for paralyzed graffiti artist TEMPT. The EyeWriter system uses inexpensive cameras and open-source computer vision software to track the wearer's eye movements. [7]Theo Watson, Chris" target="_blank">Link Sugrue[6] [8]Chris Sugrue Link and Jamie Wilkinson[7] [9]Jamie Wilkinson [10]Graffiti Markup" target="_blank">Link Language. It is a simple set of guidelines, basically an XML file format that saves x-y-time data but does it in a way that is very specifically related to graffiti so there's a drip tag and there's tags related to the size of the brush and to how many strokes you have: is it one stroke or two strokes or three strokes. The main idea is: How do you archive the motion of graffiti and not just the way graffiti looks. There are a lot of people photographing graffiti, making documentaries etc. but there hasn't been a way to archive graffiti in ways of code yet.

There hasn't been a programmatic way to archive graffiti. So this is like taking a gesture and trying to boil it down to a set of coordinate points that people can either upload or download. It is a sort of midpoint between writers and hackers. Graffiti-writers can download the software and have how-to guides for how to do this, they can digitize their tags and upload it to an open database. The 000000book-site[8] [11]Link Black Book: _A black book is a graffiti artist's sketchbook. Often used to sketch out and plan potential graffiti, and to collect tags from other writers. It is a writer's most valuable property, containing all or a majority of the person's sketches and pieces. A writer's sketchbook is carefully guarded from the police and other authorities, as it can be used as material evidence in a graffiti vandalism case and link a writer to previous illicit works._ } hosts all this data and some people are writing software for this.

Right. It starts with Graffiti Analysis. It is software written in C++ using OpenFrameworks, an open source platform designed by artists for visual applications. Right now you can download the recorder app and from that you can generate your own GML-files. And from there you can upload these files into the playback app. In the beginning that was the only open source side of the project. Programmers could also make new applications based on the software, which also happened. Last night we met Stéphane Buellet[9] [12]Stéphane Buellet, Camera Linea Link developing a calligraphy analysis project and he used Graffiti Analysis as a starting point. I find it exciting when that happens but more often people take the file-format as a starting point, and use it as a jumping-off point for making their own work. Second was the database. We had this file-format that we loosely defined. I worked with Jamie to develop the 000000book site. It is pretty nuts-and-bolts but you can click 'upload' and click on your own .gml files and it will playback in the browser. People have developed their own playback mechanisms, which are some of the first open source collaborations that happened around GML files. There is a user account and you can upload files; people have made image renderers, there are people that have made Flash players, .svg players. Golan Levin has developed an application that converts a .gml file into an auto-CAD format. The 000000book site is basically where graffiti writers connect to developers. In the middle between Graffiti Analysis and database is the Graffiti Markup Language, that I think will have it's own place on the web. But sometimes I see it as one project. One of my interests is in archiving graffiti and all of these things are ways of doing that. It is interesting how these three things work together. In terms of an OS development model it has been producing results I haven't seen when I just released source code.

We started by looking at Graffiti Analysis and L.A.S.E.R. Tag, the apps that were using graffiti motion data. From those two projects I had a lot of experience of meeting graffiti-writers as a userbase. When you meet with them, they tell you right away what pieces of the software they think are missing. So from talking with them we developed a lot of features that now are in GML like brushes, drips, line-thickness. Some people had single line tags and some people had multi-line tags so that issue came up because GML tracks both drawing and non-drawing motion so we knew that we needed in the file format to talk about pen up and pen down. I was interested in the connection points between lines also. We tried to keep it very stripped down. From the beginning we knew that people that would participate as developers or anonymous contributors were not going to be the same people that would develop a Linux core. They are students, people just getting into programming or visual programming. We wanted people to be able to double-click a .gml file and than everything should verbally make sense so it is Begin stroke. End stroke. Anyone with basic programming skills should be able to figure out what's going on.

The only thing that has to be in there is the format in x-y time scenario with some information on drawing and not drawing, everything else is bonus. So if you load in an XML-file structured like that, compliant apps will load it in. On top of that, there are features that some apps will want and others not. Keywords are, for example, a functionality that we are still developing applications for. It is there but we are looking for how to use it.

(laughs) I think in the beginning it was a very functional conversation. We were having apps running this data and I don't think we were thinking of defining graffiti when we were writing the format. But looking back, it is interesting to think about it. Graffiti has a lot of privacy issues related to it too, right? So we did discuss about what it would mean to start recording geo-located data. There are different interests in graffiti. There is an interest in visuals and in deconstructing characters. Another group is interested in it, because it is a sport and more of a performance art. For this type of interest, it is more important to know exactly where and when it happened because it is different on a rooftop in New York to a studio in the basement of someones house. But if someone realizes this data resulted from an illegal action, and wanted to tie it back to someone, than it starts to be like a surveillance camera. What happens when someone is caught with a laptop with all this data?

I see graffiti-writers as hackers. They use the city in the same way as hackers are using computer systems. They are finding ways of using a system to make it do things that it wasn't intended to do. I am not sure graffiti-writers see it this way, but I am in this position where I have friends that are hackers, playing around with digital structures on-line. Other friends are into graffiti-writing and to me those two camps are doing the most interesting things right now, but these are two communities that hardly overlap. One of the interests I have is making these two groups of people hang out more. I was physically the person bridging these two groups; I was the nerd person meeting the graffiti writers talking to them about software and having this database. Now it is not about my personal collection anymore, it is making a handshake between two communities; making them run off with each other and having fun as opposed to me having to be there all the time to make introductions.

This is an interesting conversation we should have with the graffiti-writers. A tag might be something they have been writing for more than 25 years and that will be very personal to them and the way they write this is because they've written it a million times. So at the one hand it is super-personal, but on the other hand a lot of graffiti writers have no problem sharing this data. To them it is just another tag. They feel like, _I have written this tag a billion times_ and so when you want to keep one of them, it is no big deal. I don't think the conversation has gotten as involved as it could have. You set something in motion and cross your fingers hoping that everyone plays nice and things go well and so far that is what has been happening. But you are dealing with people that are uploading something that is super personal to them and I'd be curious to see what happens in the future. The graffiti taxonomy project that I have been doing involves a lot of photos of graffiti. It is a visual studies based on characters, I am shooting thousands of photos of graffiti and I don't have an opportunity to meet with all these writers to ask them if it is OK. So I get e-mails from writers once in a while saying _Hey, you used a photograph of one of my tags_ and usually it is them feeling out where my intentions are and where I am coming from. It has taken a long time to gain the trust of the community I am working with. Usually when I am able to explain what I am doing and that everything is released openly and meant to be completely free, so far at least the people I have managed to talk toare OK with it and understand it. Initially when people see something they've made being used by other people, a lot of times it can be a point where a red flag is raised and I am assuming there are more red flags going to go up.

Not yet. Right now there is not even a _private mode_ on the 000000book site. If you upload, everything is public. There is a lot of interesting issues with respect to the license that I have been reluctant to deal with yet. Once you start talking too much about it, you will scare off people on either side of the fence. I think that will have to happen at some point but for now I have decided to refer to it as an _open database_ and I hope that people will play nicely, like I said.

It might make more sense to go for a media-related license than for a code license. Creative Commons licenses would lend themselves easily for this. People could choose non-commercial or pure public domain. Does that make sense?

Someones tag might look completely different if they had six seconds to make it, they make different decisions. In the first version of the Graffiti-Analysis project, I had one camera recorder tracking the pen and another camera behind the hand and another so you could see the full body. But there was something about tracking just the pen tip that I liked. It is an easier point of entry for dealing with the motion data than having three different video feeds.

Maybe in the keywords there will be something like: Rooftop. Brooklyn. Arrested. The most interesting part is often the stories that people tell afterward anyway. So it is an interesting idea, how to tie the story to the data. It is a design problem too. Historically graffiti has been documented many times by outsiders. The movie Style Wars[10] [13]Style Wars. Tony Silver, 1983. Link documentary that was made by outsiders that became insiders. Also, the people that have been documenting most of the graffiti are not necessarily graffiti writers. Graffiti has a history with documentarians entering into their community and playing a role but sharing the stories is something writers do internally, not as much to outsiders. How do you figure out a way to get graffiti-writers to document their stories into the .gml-files themselves, or is it going to take outsiders? How does the format facilitate that?

I am interested in open source culture being influenced by graffiti, and I am interested in open source culture influencing graffiti as well. On a big picture I would love it if the graffiti community got interested in these ideas and had more of a skill-sharing-knowledge-base. KATSU[11] [14]KATSU Link, someone I worked with in New York, has acquireda lot of knowledge about how to make tools for graffiti and he initially wasn't so much into sharing them, because graffiti-writers tend to save that knowledge for themselves so that their tags are always bigger and better (laughs). Talking to him I think I convinced him to write tutorials on how to make some of these tools. On the street art side there is Mark Jenkins[12] [15]Mark Jenkins tapesculptures Link, he has this technique of making 3D objects that exist within the city and we had a lot of conversations too. There are many ways tech circles and open source circles can come together with people that are making things outside, with their hands. I think graffiti can learn from that. In the end people would be making more things outside which would be a good thing.

Interesting. I think the work I am doing is such a tangent! In general, for something that is decidedly against the rules, the culture of writing graffiti often has a rigid structure. To people in that community what I do is a blip on their radar. I am honored when I get to meet graffiti-writers and they are interested in what I am doing but I don't think it will change anything in what is in some ways a very strict system. And I don't want that either. I like the fact that they found a way to make spray-paint and markers change the way each city in the world looks. They have the tools they need. Digital projectors will not change that. Graffiti-writers still like to see their names projected at big scales in new ways but it is not something they really need (laughs).

For the people on the technology side, it is an easy jump. To think about hacking software systems and than about making things outside. I see that with the Free Art and Technology Group[13] [16]The Free Art and Technology (F.A.T.) Lab is an organization dedicated to enriching the public domain through the research and development of creative technologies and media. Release early, often and with rap music. Link projects in the city, it takes little to come up with great ideas. I also see that in the class I teach, Urban Hacking. There is already a natural overlap.

It is really about the idea of hacking. The first assignment in the class is not to make anything, but simply to identify systems in the city. What are elements that repeat. Trying to find which ones you can slip into. It has been happening in graffiti forever. Graffiti in New York in the 80's was to me a hack, a way to have giant paintings circulating in the city... There is a lot of room to explore there.

Recently I released a piece of software that translates a .gml file and translates it into a .stl file, which is a common 3D format. So you can basically take a graffiti gesture and import it into software like Blender. I used Blender because I wanted to highlight this tool, because I want these communities to talk to each other. So I was taking a tag that was created in the streets of Vienna and pulling it into Blender and in the end I was exporting it to something that could be 3D printed, to become something physical. The video that I posted intentionally showed on-line showed screenshots from Blender and it ended up on one of the bigger community sites. I only saw it when my cousin, who is a big Blender user, e-mailed me the thread. There is about a hundred dedicated Blender users discussing the legitimacy of graffiti in art and how their tools are used[15] [18]Link ; pretty interesting but also pretty conservative.

It doesn't surprise me that much. Graffiti is hard to accept, especially when we are talking about tags. So the only reason we might be slightly surprised by hearing people in the Open Source community react that way, is because intellectual property doesn't translate always to physical property. Writing your name on someone's door is something people universally don't like. I understand. For me the connection makes sense but just because you make open source doesn't mean you'll be interested in graffiti or street art or vice versa. I think if I went to a Blender conference and gave a talk where I explained sort of where I see these things overlap, I could make a better case than the three minute video they reacted to.

Essentially GML records x-y-time data. If you talk about what it functionally does, it is probably more related to gesture than it is to graffiti. There is nothing at the core specifically related to graffiti. I am interested in branding it in relation to graffiti and to get people to talk about open source where it is traditionally not talked about. To me that is interesting. It is a way to get people excited about open data, and popularizing ideas about open source.

We haven't really entered into that. Why would you need a license on a file-format?

That sounds good. Actually it would be interesting for the project, if someone would try to license it. Legal things matter, but for the things I do, I am most of all interested in getting the idea across.

Gesture recognition to help out the police?

It is hard for me to imagine something. But one thing is interesting for example, how GML is used in the EyeWriter project. It has not so much to do with gesture, but more with how you would draft in a computer. TEMPT is plotting points, so the time data might not be so interesting but because it is in the same format, the community might pick it up and do something with it. All the TEMPT data he writes with his eyes and it is uploaded to the 000000book site automatically. That allowed another artist called Benjamin Gaulon[16] [21]Benjamin Gaulon, Print Ball Link who I now know, but didn't know at the time, to use it with his Print Ball project. He took the tag data from a paralyzed graffiti writer in Los Angeles and painted it on a wall in Dublin. Eye-movement translated into a paint-ball gun ... that is the kind of collaboration that I hope .gml can be the middle-point for. If that happens, things can start to extrapolate on either end.

I'll answer that in a second, but you reminded me of something else: because EyeWriter was .gml based, a lot of the collaborations that happened with people outside of the project were .gml related, not EyeWriter related. So we did have artists like Ben and Golan take data drawn by TEMPT and do completely different things which made TEMPT a collaborator with them in a way. The software allowed him to share his work in a format that allowed other people to work with him. The wish-list came out of the fact that I was working on a graffiti-related project that had a lot of use but not a lot of innovation. Not so many people were using it in ways I wasn't expecting, which is something you always hope of course. By saying: _Here's the things I really would like to happen_, things started to happen. I have been surprised how that drove momentum. Something similar I hope will happen to the work we will do together in the next months too!

We are planning to make a dedicated community page for the graffiti markup language which is one of the three points of the triangle. The second step would be a new addition to the wish-list, a challenge with a prize associated to it which seems funny. The project I'd like to concentrate on is making the data collection easier so that graffiti writers can be more active in the upload sense. Taking the NASA development model: Can you get into orbit on this budget?

If I go out with a graffiti-writer, I'm stuck standing with a laptop and a camera facing the wall and then the graffiti writer needs to have a really bright light attached to the writing device which is a bit counter-intuitive when you are trying to do something without being seen (laughs). It could be infrared by the way, that could be the first step but then security cameras would still pick it up. The design I am focusing momentum on is a system that's easier. A system that can work without me there, without having to have a laptop there. The whole idea is that it would be a natural way to get good data, to document graffiti without a red-head holding a laptop following you around the whole time!

Paris, December 2010

It is not over. It was easy to launch; I tried to coincide it with the launch of the website and there were a couple of things going on at the same time. The launch helped spread the word about this file-format, and people making projects, and vice versa.

No. There are a couple of people that got really close. The interesting thing that is happening with the challenge is something that is also happening to other high barrier projects: You end up speaking to the people you already work with the most. I have a hard time figuring out to some extent what is really happening, but the things I hear, of people making progress, is people that are close to me. It reminds me of the EyeWriter project where people that are to dip their toes into this, are already in the friend group, or one level removed. They are pretty high level programmers. I didn't really think that actual money would be such an incentive but more that it would make the challenge feel serious, more in the sense of an organization that has some kind of club behind it. If you solved one of the design problems by the Mozilla community you could receive kudo's from the community, but if you solved one of my projects, you don't really get kudo's from my community, do you? Having the money associated makes it this big thing. At Ars Electronica and so on, it got people talking about it and so it is out there. That part worked. Beyond that it has been a bit hard to keep the momentum. Friends and colleagues send me ideas and ask me to look at things, but people I don't know are hard to follow; I don't think they are publishing their progress. There is a hackerspace in Porto that has been working on it, so I see on their blog and Twitter that they are having meetings about this and are working on it.

Maybe. Kyle[19] [24]Kyle McDonald Link has been thinking up ways to do it and I know he wanted to use an optical mouse, and then this a friend Michael [20] [25]Michael Auger Link has been using sensors, and he ran into a software problem but had the hardware problem more or less solved. And then Kyle, a software expert, has been running into hardware problems and so I kind of introduced them to each other over e-mail so I don't know if they are working on it together.

I don't care, but I don't know if the candidates would consider splitting the prize! I know Michael has already spent a lot of money because he has been buying Arduino's and other hardware. He wants to make a cheap version to solve the problem and then make another one that costs 150 euro on top of the price limitation to make it easier to use. He is spending a bunch of money so even if he wins, it is going to get him only out of the hole and he will not have much left. Actually, Golan[21] [26]Golan Levin Link had an idea for an iPhone app that he wants to make but I am not sure it solves it.

He is really interested in making something where you do not need to meet with the graffiti-writer. His idea was that if you could take a photo of it on the wall, and then with your finger you guide it for how it was written. It has an algorithm for image processing and that combined with your best guess of how it was written would be backed out in motion data. But it is faked data.

Yes it is and I would love it if he would make it but I am not going to let him win with it (laughs). I understand why he wants to do it; especially if you are not inside the graffiti community, your only experience is what you see on the wall and you don't know who these people are and it is going to be almost impossible to ever get data for those tags. If you don't have access to that community you are never going to get the tag of the person that you really want. I like the idea that he is thinking about getting some data from the wall as opposed to getting it from the hand.

With the EyeWriter project, we were thinking a lot about that; how to spur development. I think I would not have done a competition with the EyeWriter. Making it fun, that is what makes it happen. If it would be a really serious amount of money, with people scraping at each other, fighting each other ... For me, the fact that there is prize money makes something that is already ridiculous in itself even more funny. To have prize money for such a small community of people that are interested in coding {} in graffiti. I'm not seriously thinking that we can spur development with this kind of money. To use the EyeWriter as an example, we've had money infusions from awards mostly and we had to think about how we could use that money to get from point A to point B. That's also a project where we had very definable design goals of what we wanted to reach, especially between the first version and where we are now with the second version.

We are not talking about a ton of money here, 10 to 20.000 euro, and we tried to get as far as we could. We got almost no work done between the meetings in LA but if we flew in, it was OK to take a week out of our schedules and really hammer at it. We were trying to think how we could do the same thing for people that we wanted to work with and who we had met in conferences. So that is how we thought of spending that money. The other way we use money in the EyeWriter project is that we buy people kits. We know a few people that are interested in hacking on it but they don't have the hardware. Not that they are so expensive, but Zach wants to buy twenty or thirty unpackaged kits and he has interns working with him in New York helping to build them. So we have these systems ready so as soon as someone wants to get hacking on it, we can mail them a working system that they can just plug in and they don't have to waste their time ordering all these parts from all these websites all over China. And when they are done, they just send it back.

I think the forum is the obvious thing that did not work. I have friends working on Open Frameworks, it is headed primarily by Zach and Theo. When you see that forum, it is very involved. It is a deep system, with many different libraries and lots of code flying around. GML is really not large enough. I think what makes sense for this project is when I post news about the project, I see it ripple in Google Alerts. For people working on it, having a place where these things show up is already a lot. The biggest success is the project space, to see all the projects happening.

A project I like, is kml2GML[24] [29]Yamaguchi Takahiro Link by a friend from Tokyo. He was gathering GPS data riding his bike around various cities, and building up a font based on his path. I like projects like this, where someone takes a work that is already done and just writes an application to convert the data into another format. To see him riding his bike played back in GML was really nice. It is super low barrier to entry, he already did all the hard work. I like that there is now a system for piping very different kinds of data through GML.

Yeah. This is maybe a tangent but depending on how someone solves the GML-challenge... I was discussing this with Mike (the person that is developing the sensor based version). He was thinking that if you would turn on his system, and leave it on for a whole night of graffiti writing, you would have the gestural data plus the gps data. You could make a GML-file that is tracking you down the street, and zoom in when you start making the tag. Also you would get much more information on 3D movement, like tilt and when the pen is picking up and going down. Right now all I am getting is a 2D view through video data. I am really keeping my fingers crossed. But he ran into trouble though.

I have my doubts about using these kind of sensors, because drift is a problem. When you start using these sensors too long, it tends to move a little bit. I think he is working within a 0.25 inch margin of error right now, which is right on the edge. If you are recording someone doing a big piece, this is not going to ruin my day too much but if you record a little tag than it is a problem. The other problem is that you need to orient the system before you start tagging. It needs to know what is up and down, you have to define your plane of access. I don't really understand this 100% but he thinks he can still fit it all within the ten second calibration requirement, he's thinking that each time you come to a wall, you tap once, you tap twice and tap a third time to define what plane you are writing on and that calibrates the 3D space. Once you have that calibration done, you can start writing. It is not as easy as attaching a motion sensor. The problem is hard.

Theo was interested in using the iPhone to record motion data in GML, but also to save the coordinates so you could try it into a Google Earth or something but he had trouble with the sensitivity of the sensor. Maybe it is better now but you needed to draw on a huge scale for one letter. You could not record anything small.

I know. I have just been experimenting with mounting cameras on spray-cans. A tangent to GML, but related. It is not data, but video.

Video is annoying in that it is computationally expensive. In Brazil[25] <span class="NOTE">[30] Graffiti Analysis: Belo Horizonte, Brazil 2010 Link not very precise. Kyle thinks he might be able to back out GML data from videos. This might solve the challenge, depending on how many cameras you need and how expensive they are. But so far I have not heard back from him. He said it needs three different cameras all looking at the wall. I mean: talk about computationally expensive! He likes video-processing, he knows some open source software that can look for similar things and knows how to relate them. To me it seems more difficult than it needs to be (laughs).

I am getting more into video myself. I get more enjoyment from capturing the data than from the projections, like what most people associate with my work.

In part because it stays new, I've been doing those projections for a while now and I know what happens at these events. For a while it was very new, we just did it with friends, to project on the Brooklyn bridge for example. Now it has turned into these events where everyone knows in advance, instead of just showing up at at a certain time ate a set corner. It has lost a lot of its magic and power. Michele and I have done so many of these projections and we sort of know what to expect from it, what questions people will ask. When I meet with graffiti writers, that almost always feels new to me. When we went to Brazil, we intentionally tried to not project anything but to spend as much time as possible with writers. Going out with graffiti writers to me always feels right.

To me documentation is interesting. I don't know where all of this is going right now, I am just trying to get the footage; I put these pieces together showing all this movement but I don't really know what the final project is. It is more about collecting data so I am interested in having video, audio and GML that can be synced up, and the sound from these microphones is something to do something with later. This is research for me. I like the idea of having all this data related to a 10 second gesture. I am thinking that in the future we can do interesting things with it. I am even thinking about how the audio could be used as a signal to tell you what is drawing and what is not drawing. It is a really analog way of doing it, but in that way you don't need a button where you are getting true and false statements for what is drawing and what is not drawing; you can just tell by the sound: tfffpt ... tfffpt.

I got started doing this because I love graffiti and this is a way to get closer to it again. Like getting back out to the streets and having very personal relationships to the graffiti-writers and talking to them, and having them give feedback. I think that is how the whole challenge started. It didn't start because I was projecting, but because I was out on the street and testing the capture, having graffiti-writers nearby when it is happening. It feels like things are progressing that way.

I have been missing out on that piece. It is a small amount of time we have, and I am already trying to get so much. I am setting up a camera that shoots straight video from a tripod, I am capturing from the laptop and I am also screen-casting the application, my head is spinning. One reason I screwed up this footage in the beginning is because with all these things going on I forget to turn on some things. Maybe someone will solve this challenge.

In the back of my head I am thinking this will become a longer documentary. I like to experiment with documentation, whether that is in code or with video. I do think that there is this interesting connection between documentation and graffiti and how these two things overlap. I am always thinking about documentation. The graffiti-writer that was in Vienna[26] [31]momo3010 Link showed me a video that was amazing. It was him and a friend going out on a sunny day at 15:30 in the afternoon with two head mounted cameras, bombing an entire train and you hear the birds singing and you only experience it by these two videos that are linked. There are interesting constraints: your hands are already full, you don't want peoples' faces on camera so the head-mounted cameras were smart. Unless you walk in front of a mirror (laughs).

I like that. Even doing the challenge is in a way a reflection on this, how I am fighting to get GML back to the streets somehow, it has a natural tendency to get closer to the browser, to the screen, and my job is to get it back to the street. It is so sexy and fun and flashy and that is important too. My job is to keep the graffiti influence on it as large as the other part.

I haven't looked at the standard for a while now.

Maybe using an XML-like structure was a bad idea? Maybe if I had started with a less code-based set of rules? If the files were raw video, it would encourage people to go outside more often? By picking XML I am defining where the thing heads in a way. I think I am OK in the role of fighting that tendency. It is not just a problem in GML but with a lot of work I have been doing with graffiti and technology and even way back with Graffiti Analysis, before GRL (Graffiti Research Lab), the idea was always to keep the research very close to the people doing graffiti. I was intentionally working with people bombing a lot and not with graffiti celebrities. I wanted to work with who's tag was on my mailbox, who's tag do I see a million times when I walk down the street. Since then a lot has happened, like with more popular projects such as L.A.S.E.R. Tag, and it goes almost always further away from graffiti. Maybe that is a function of technology. Technology, or the way it is now, will always drift towards entertainment uses, commercial uses.

It reminds me of an interview[28] [33]Interview with Francois Chastanet Link Chastanet who wrote a book[29] [34]Francois Chastanet, Cholo writing: Latino gang graffiti in Los Angeles. Dokument, 2009} about tagging in Los Angeles. He explains that the Gothic lettering is inspired by administrative papers!

A friend in Hong Kong, MC Yan, loves the Graffiti Analysis project because it shows the order in which he is writing and he likes to play with that. So he writes words in different order than people are used to and so it changes the meaning. People can not only watch the final result, but also the order which is an interesting part of the writing process. The brush, the angle, direction: depicting motion! In the beginning of the Graffiti Analysis Research project I was very against projection, because I felt that was totally against the idea of graffiti. I was presenting all of these print ideas and the output would be pasted back into the city because I was against making an impermanent representation of the data. In the end Zach said, you are just fighting this because you have a motion project and you want to project motion and then I said alright, I'll do a test. And the tests were so exciting that I felt OK with it.

I think the next challenge I'd like to run is about recreating the data outside. I've been thinking about these helicopters with embedded wireless camera's, have you seen them? The obvious thing to me would be uploading a GML-file to one of these helicopters that is dripping paint on a rooftop. Scale is so important, so going bigger is always going to be better. Gigantic rooftop tags could be a way to tie it back to the city, give it a reason? I am thinking of ways to get an edge back to the project. The GML-challenge is already a step into that direction; it is not about the prettiest screensaver. To ask people to design something that is tying back to what graffiti is, which is in a way a crime. I think fixing the data capture is the right place to start, the next one could be about making marks in the city. Like: the first person to recreate this GML-tag on the roof of this building, that would be fun. The first person that could put this _Hello World_ tag onto the Brooklyn bridge and get a photo of it gets the prize. That would get us back to the question of how we leave marks on the surface of the city.

That kind of stuff is interesting. It would help graffiti-writers to design tags maybe? To get my feet wet, I designed a tag once, and it was so not-fun to write! I was thinking about a tag that would look different and that would fit into corners, I was interested in designing something that wasn't curved; that would fit the angles of the city, hard edges. So I had forgotten all my research about drafting and writing. I think I stopped writing in part because the tag I picked wasn't fun o write. For a font to work like writing, it is not just about possible connections between lines. You'd need another level in the algorithm, the way the hand likes to move.

I was trying to think of how the challenge could be gamed... I did not want to get into a situation where we were getting stressed out because some smart hacker found a hole in the brief, and bought a next generation iPhone that somehow just worked. I didn't want to force people to buy expensive equipment. This line was more about covering our own ass.

The graffiti writer must be able to activate the recording function alone (i.e., without assistance from anyone else).

Thinking selfishly, I screw up on documentation a lot because I have too many hats. When I'm going out doing this, I am carrying a laptop, a calibration set up, I also have one video-camera on me that is just documenting, I have another one on a tripod, and I am usually screen capturing the software as it processes the video-footage because it tells another story. I screw up because I forget to hit stop or record. If the data-capture just works, I can go have fun getting good video-footage.

This came out of me dealing with the current system. It feels wrong that it takes ten to fifteen minutes to get it running. Graffiti is not meant to be that way. This speaks to the problem of the documentation infringing on the writing process, which ideally wouldn't happen. The longer the set-up takes, the more it is going to influence the actual writing. It is supposed to be a fly on the wall.

(laughs) If I had to do that one again, I would have put that in Bonus category actually. That's a difficult question to ask. What does secure mean? It seems a bit unfair, because it doesn't fit in to the way graffiti is currently documented. There's not a lot of graffiti-writers that currently are shooting encrypted photo's and video's, right? But whatever bizarre format comes out from the sensor will help. I don't think that the NYPD will have time or make the effort to parse it. They'd just have a file with a bunch of numbers. Time stamped GPS-coordinates would be more dangerous.

In most cases it is hard to convict someone on the basis of a photo of a tag that you would tie to another tag. For good reasons, because if it is a crew name for example, all of a sudden you are pinning one tag on a person that could have been written by twenty people. This came up in a trial in DC when an artist named BORF got arrested. He had written his name everywhere, completely crushed DC and his trial was a big deal. This issue came up and they argued that BORF was a collective, not an individual. Who knows if that's true, there were a lot of people around him, but how do you really know?

You mean it would not be just the image of a tag but more like signing at the bank?

This again should probably have been a bonus item. Wouldn't it be awesome if you could go home and log in and flip a one to a zero and the evidence goes up in smoke? One graffiti writer friend told me: _If the police comes, just smash the camera as hard as you can!_ It's a silly idea, but it shows that they are thinking about it.

Yes.

So this is where Muharrem is going to run into trouble. His solution interferes. Not that much if you are just working in front of your body space. But the way most writers write is that they are shuffling their feet a lot, moving down the wall. Should it have said: _Graffiti-writer should retain access to feet functionality_? This point should be at the top almost.

I want to have on record: I love his solution! There's a lot in his design that is making us more aware of what's happening in the creation of a tag. One thing that he is doing that is not in the specs, is that he is logging strokes, like up and down. When you watch him using it, you can see a little light going from red to green when the fingers goes on and off the spray-paint can. When you watch graffiti, it is too small of a movement to even notice but when you are seeing that, it adds another level of understanding of how they are writing.

All motion data should be saved using the current GML standard (Link

It wouldn't be exciting if only ten people can make this thing. This tool should not be just for people that can make NASA-qualified soldering connections. Ideally it should not have any soldering. I always thought of a soldering iron like a huge barrier point. I'm all for duct-taped electrical connections.

Yes. Security-wise that is questionable, but the nerd in me would just love it. You could get really interesting data about a whole night of writing. You could see a bigger story than just that of a single tag. How long did it take to gain entry? How long were they hiding in the bushes? These things get back to graffiti as a performance art rather than a form of visual art.

Paris, November 2011

I think in terms of the actual challenge, the main update is that since we extended the deadline and made another call, I got an e-mail right on the deadline today from Joshua Noble[32] [39]Joshua Noble Link with a very solid and pretty smart proposal that seems to solve (maybe unfortunately for Muharrem) a bit more of the design-spec. It does it for cheaper and does it in a way that I think is going to be easier to make also. His design solution is using an optical mouse and he changed the sensors so it has a stronger LED. He uses a modified lens on top of a plastic lens that comes on top of a mouse, so that it can look at a surface that is a set distance away. It has another sensor that looks at pitch, tilt and orientation, but he is using that only to orient, the actual data gets recorded through the mouse. It can get very high resolution, he is looking at up to a millimeter I guess.

I think he gets away with more because his solution is only for spray-paint and once you are writing on that scale, even if you are off a few centimeters, it might not ruin the data. If you look at the data he is getting, it actually looks very good. I don't think he has any numbers on the actual resolution he is getting but if you were using his system with a pen, I think it would be a different case. I like a lot of his solution too, it is an interesting hack. It is funny that two of the candidates for the prize are both mouse hacks. One is hacking a mechanical mouse and the other an optical mouse.

Yes. When I first was working on graffiti-related software, the whole reason I was building Graffiti Analysis as a capture application was because I did not want to hand graffiti-writers a mouse (laughter). I had done all this research into graffiti and started to be embedded in the community and I knew enough about the community that if you were going to ask them to take part in something that was already weird, you could not give them a mouse and expect any respect on the other end of that conversation. They respect their tools, so the reason I was using camera-input was because I wanted to have a flexible system where they could bring in anything and I could attach a device to it. Now I am coming back to mice finally.

I think it was a good experiment, I am not sure how clever it was. To take a piece of culture that a lot of people don't even look at, or look at it and think it is trash, to invest all this time and research and software expertise into it makes people think about the graffiti practice and what it actually is. The cash prize does something similar. It attaches weight to something that most people don't even care about. Even having the name of an organization like Constant attached to it is showing that I am really serious about this. In that sense it is different than a wishlist. I just read the Linus Torvalds[33] [40]Torvalds, Linus; David Diamond (2001). Just For Fun: The Story of an Accidental Revolutionary. New York, New York, United States: HarperCollins.} biography, and I liked his idea that fun is part of innovation, right? In a programming sense, it is scratching a personal itch. The attachment of a prize is more to underline the fun aspect than anything else.

I understand what you mean.

Even Open and Free are in competition! In a project like White-Glove Tracking for example, the most popular video I had not made and it did not have my name on it but personally I still felt a part of it. I think when you are working in open systems, you take pride when a project has wings. It is maybe even a selfish act. It is the story of me receiving some art-finding and realizing that I am not the best toolmaker for the job. Who ever manages to win the prize gets all the glory, but I'm still going to feel awesome about it.

Yes, I thought that was an interesting point. I've been in similar conversations on listservs with artists in the OpenFrameworks, Processing and visual programming communities. What are the open pieces? It makes sense to share libraries, but if I make a print from a piece of code, do I then have to share the exact source and app for how that exact print was made? What does it mean when I am investing money in a print, and it is a limited series but I'm sharing the code? The art world is still based on scarcity and we're interested in computers that are copy-machines. I see both sides of the argument and I am still trying to see how I fit into it. It gets trickier when you are asked to release a piece rather than a tool. If you are an open source artist and you make a toolset, that is easier to share because people use that to make their own things. But then an artist gets asked: how come I can't get the file of that print? I think that is a really hard question.

Graffiti has an interesting relationship to archiving. On the one hand, many graffiti writers think: Now that tag's done, but I've got another million of them. While others do not want people painting over them, the city or other graffiti writers. Also if a tag has been up there for a few years, it acquires more reverence and it is even worse when it is painted over. But I think that GML is different, it is really more similar to a photo of the tag. It is not trying to be the actual thing.

I've seen cases where both parties have been happy, like when [42]Yamaguchi Takahiro used some GML data from KATSU and piped it into Google Maps, so he was showing these big KATSU tags all over the earth which was a nice web-based implementation. I think he was doing what a graffiti writer does naturally: Get out there and make the tag bigger but in different ways. He is not taking KATSU-data from the database without shining light back on him.

I have my reasons for which I would and would not use certain pieces of data in certain contexts, but I like the fact that it is open for people that might use it for other things, even if I would not push some of those boundaries myself.

It is interesting to be worried about copyright on something that is illegal, things you can not publicly claim ownership of.

For that to happen, a larger community would have to get involved. It would need to be Gesture Markup Language, and a community other than graffiti-writers would need to get involved.

It would be so interesting to have a cribs moment with some gangsta or rapper who came from that, and he's sort of showing off his stuff and he has this machismo about him. Not necessarily directly mysognistic but a macho kind of character and then take a nerd and have them do the same.

Obviously some rappers and some nerds, I mean that's one of the beauties-{}-I mean its a global movement, you can't help but have diversity-{}-but if we're just speaking in generalizations?

Right.

Breaking currently existing software? I hope not.

When the spec was being made of course it wasn't just me, it was a group of people debating these things and of course nobody wants things to break. The idea was that we tried to get in as many things as we could think of and have the base stay kind of what it was with the idea that you could add more stuff into it. It's easy enough to do, of course its not a super-rigid standard. If you look at what the base GML file is, the minimum requirements for GML to compile, its so so stripped down. As long as it just remains time/x-y-z, I don't think that's going to change, no. But I'm also hoping that I'm not gonna be the main GML developer. I'm already not, there's already people doing way more stuff with it than I am.