tag:blogger.com,1999:blog-71808922010-05-10T21:40:18.976-04:00a life of codingtrust your technolustynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.comBlogger68125tag:blogger.com,1999:blog-7180892.post-21811263512559667412010-02-26T10:26:00.003-05:002010-02-26T13:41:40.257-05:00Looking at the Facebook PatentI Am Not A Lawyer.<br /><br />Facebook was recently granted a patent for "<a href="http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=/netahtml/PTO/search-bool.html&r=1&f=G&l=50&co1=AND&d=PTXT&s1=Facebook.ASNM.&OS=AN/Facebook&RS=AN/Facebook">Dynamically providing a news feed about a user of a social network</a>". This has generated much <a href="http://edition.cnn.com/2010/TECH/02/26/facebook.patent/">news</a>, and has many <a href="http://www.allfacebook.com/2010/02/facebook-feed-patent/">people</a> <a href="http://www.readwriteweb.com/archives/facebook_granted_patent_on_the_news_feed_-_this_co.php">complaining</a> about <a href="http://www.avc.com/a_vc/2010/02/more-patent-nonsense.html">software patents</a>. Unfortunately most comments look no further than the title.<br /><br />The title is irrelevant. In fact, most of the text of a patent is irrelevant. Whenever you are presented with a patent, you should start with the independent claims. The claims are what you litigate (sue over), and the independent claims are the broadest. If you create a "social network news feed" which does not match one of the claims in Facebook's patent exactly (or equivalently), this patent has no bearing on you.<br /><br />The three independent claims (1, 16, 24) are almost exactly the same, differing in the embodiments of a "method", a "system", and a "computer readable medium" and changing the grammatical structure to fit. Here is the first one, the others are trivially the same:<br /><br /><ul>1. A method for displaying a news feed in a social network environment, the method comprising: monitoring a plurality of activities in a social network environment; storing the plurality of activities in a database; generating a plurality of news items regarding one or more of the activities, wherein one or more of the news items is for presentation to one or more viewing users and relates to an activity that was performed by another user; attaching a link associated with at least one of the activities of another user to at least one of the plurality of news items where the link enables a viewing user to participate in the same activity as the another user; limiting access to the plurality of news items to a set of viewing users; and displaying a news feed comprising two or more of the plurality of news items to at least one viewing user of the predetermined set of viewing users.</ul><br />Infringing this claim requires (a) monitoring activity, (b) storing it in a database, (c) generating new items from the activity that you plan on showing users, (d) attaching a link to participate in the same activity, (e) limiting who can see them, (f) displaying more than one news item to a user who did not specifically ask for it.<br /><br />I would say that any basic security monitoring and alerting system would be sufficient prior art, except for (e). The claim specifically mentions attaching a link from "<i>attaching a link associated with at least one of the activities of another user to at least one of the plurality of news items where the link enables a viewing user to participate in the same activity as the another user</i>". This seems an oddly specific limitation on the claim, which leads me to believe (given the massive prior art list) that the claim was not accepted without it. So, if you write a news feed that does not attach <b>at least one</b> link that does <b>exactly this</b>, you are not infringing this claim.<br /><br />PS: You should should have realized by now that this patent has nothing at all to do with twitter. Or really most social network feeds.<br /><br />[ more comments at <a href="http://news.ycombinator.com/item?id=1153395">Hacker News</a> ]<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-2181126351255966741?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com0tag:blogger.com,1999:blog-7180892.post-29374251108435565002010-01-28T13:19:00.004-05:002010-01-28T13:52:44.352-05:00Skirt the AppStore: Run Unsigned Code on a Stock iPhone / iPad<p>Another iPhone OS product, another flame-war over Apple's "iron fisted control". I think the whole argument is pretty silly, and to hammer it home I gave this post a title that's sure to anger lots of people. Why? Because what I'm about to say is (a) obvious, (b) correct, and (c) still not what people want to hear.</p><p>You can run arbitrary code on an unmodified iPhone, and the same will be true of the iPad. You will still have certain restrictions (only one app at a time, no access to the data in other apps, and limited access to hardware), but rarely do people complain about those. What people do say is that "Apple will stop me from doing X, because X interferes with their business model, or angers the carriers." Examples are Skype over 3G, Google Voice... really, there aren't that may reasonable apps that have been turned down on the AppStore, but as I said, this is a flame-war and "reasonable" isn't part of the vocabulary.</p><p>You can run whatever your little heart desires on your iPhone/iPad, and it only costs you $99 more. This product "upgrade" is called the Developer Program, and has been available for quite some time. After paying the fee, Apple lets you download a piece of software that lets you run almost anything that you like (I already listed the restrictions) on your iPhone. Done!</p><p>Whats truly amazing is that <i>this is always how desktop development has always been done, except on Linux and Mac OS X</i>. Pay your money, get your compiler, write your software, run your software. The Linux and Mac OS X crowd is up in arms because they have had GCC for at least 9 years, and $99 seems like a lot compared to free. Where things differ is what happens after you have written a piece of software that you like and runs correctly. Traditionally, you make the binary available and people can click it and have it run (insecurely) on their computer. With the iPhone OS, you send it to Apple, they make sure that it fits some minimum criteria, and now you can require people to pay you to use this software. The end user even gets a convenient update mechanism, and a way to re-acquire your application without re-paying should it be deleted.</p><p>Want to be a revolutionary? Here's some advice: stop complaining, join the developer program, and make something people want.</p><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-2937425110843556500?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com1tag:blogger.com,1999:blog-7180892.post-67265788535463536882010-01-21T14:53:00.002-05:002010-01-21T15:18:40.842-05:00The Most Amazing Error Condition<p>I stumbled on a dumbfounding message today. It said: "The application Preview can't be opened -10810." I saw another one later that said: "The application TextEdit can't be opened -10810." I have seen strange errors when upgrading (I assume due to pieces of libraries cached in memory differing from the remaining code on disk), but I had not upgraded recently. Thinking that I may have run out of disk space, I decided to check "df -h". Opening a new Terminal window just said: "Could not open a new pseudo-tty."</p><p>"Huh. Thats odd..." A search brought up some old Linux threads mentioning a maximum number of processes-per-user. I couldn't think of any reason why I would have too many processes, so I went to an existing shell, typed "ps -ax" and got "fork: Resource temporarily unavailable." Further adding to the awesomeness of the situation. But, I had a possible out - if there were indeed too many processes running, I could sacrifice some and the problem would (temporarily) go away.</p><p>I did manage to run a "ps -ax", and I saw a large number of:</p><pre> 502 4344 ?? 0:00.00 (Google Chrome He)<br /> 502 4432 ?? 0:00.00 (Google Chrome He)<br /> 502 4433 ?? 0:00.00 (Google Chrome He)</pre><p>... zombie Google Chrome processes! Hah! I have been running Chrome for long periods of time, and its process management system is not functioning correctly. Close Chrome, problem solved.</p><p>Man, that was a weird one.</p><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-6726578853546353688?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com1tag:blogger.com,1999:blog-7180892.post-36200430276562526542009-11-19T16:54:00.007-05:002009-11-19T23:46:32.595-05:00Paul's Mistake (Be Strong, Not Whiny)Paul Graham recently commented on the state of Apple's AppStore and their treatment of developers. The comment seemed uncharacteristic of him, dedicating a lot of time to chastising Apple for their actions and claiming that this will substantially tarnish Apple's reputation. I respect Paul, but feel that his tone is wrong. Only time will judge whether Apple's treatment of developers will keep them from being successful. Instead of complaining, we should be analyzing how Apple has been successful and attempt to reproduce that on our own. Its the longer, harder path, but it will make all the difference.<br /><br />(continuing with the previously titled "Developers: You Are Whats Wrong With the iPhone AppStore")<br /><br />You pounded your fists because Apple didn't have a "real" SDK, so Apple created one. Then you pounded your fists because the approval process is too slow, so Apple hired a bunch of noobs. Now you're pounding your fists because the newbies aren't consistent in their execution.<br /><br />Stop. Stop making iPhone apps. Stop complaining about Apple. Stop using their phone. Stop.<br /><br />I am a long time Mac user, and I will tell you that nobody cares about your app, least of all Apple. I didn't buy the iPhone because of you, I bought it because of Apple. Apple put a bunch of cool things on it, and they all work (mostly) great. Most AppStore apps sell for a dollar, because thats about all they're worth. (BTW: please disprove this by creating something valuable.)<br /><br />Since Apple made a great phone, they have become popular. But Apple isn't good at popular. Popular people have to bend to the will of others to stay popular, and that just isn't Jobs, and it isn't me either. Apple wants to be the best, and you have to be a little elitist to do that. You have to somewhat ignore what everyone wants and concentrate on what is best. Personally, I'm okay with this, and so are most of the people who bought an iPhone.<br /><br />When Apple releases a tablet, will your iPhone app work on that? Of course not. What about Google's Droid? Not there either. Your apps will be rewritten (and re-tested) on every platform there is. The reason that we were all making web apps before the iPhone was that web apps actually work everywhere. JavaScript+HTML is the data interchange format for executable code (delivering what Java was supposed to). Don't like Apple's phone? Get a new one, your email, photos and web applications will continue to work there. Man that sounds nice, doesn't it.<br /><br />If Apple is pissing off developers, I personally think this is great. Instead of building yet another iPhone app that Apple has to test and only your friends will ever use, you'll think about building your own hardware, or an app for another phone, or a web app that will be usable on any device for decades. Better yet, build a WebAppStore that works exactly like the AppStore, but for web applications.<br /><br />You can pound your fists all you want, but only a couple of your peers are even listening.<br /><br />[Discuss at <a href="http://news.ycombinator.com/item?id=951384">Hacker News</a>]<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-3620043027656252654?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com13tag:blogger.com,1999:blog-7180892.post-38186175385781523602009-09-02T16:33:00.007-04:002009-11-19T17:52:36.658-05:00Cleaning MacPorts DependenciesSometimes MacPorts gets a little carried away with dependencies. I recently tried installing mercurial on 10.6, and found MacPorts <a href="http://joi.ito.com/weblog/2005/03/05/yak-shaving.html">yak shaving</a> various X11 libraries. After deciding that this yak didn't need to be shaved, I cancelled the install and downloaded the Mac OS X <a href="http://mercurial.berkwood.com/binaries/Mercurial-1.3.1-py2.6-macosx10.6.zip">mercurial installer</a> (<a href="http://mercurial.berkwood.com/">index</a>).<br /><br />Unfortunately, canceling a port install leaves behind extra packages. Which packages? Good question. I knew that I had asked for some packages (libidl, graphviz, bash-completion), but I didn't know which packages were only installed as dependencies of my half completed mercurial request. Given a package, <code>port</code> can find the packages that are dependent on it. Hmm, this sounds like some programming homework...<br /><br />Given a map of package names to a list of packages dependent on that package:<br /><ol><li>find packages with no dependencies, which were not explicitly installed by the user</li><li>if there are none, you are done</li><li><ol><li>add them to a list of packages to remove</li><li>remove their map entry</li><li>remove them wherever they appear as dependent on another package</li></ol></li><li>recurse!</li></ol><br /><br /><script src="http://gist.github.com/179943.js"></script><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-3818617538578152360?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com2tag:blogger.com,1999:blog-7180892.post-33991755459629335352009-08-10T14:29:00.005-04:002009-08-10T16:23:17.897-04:00Google Wave isn't Email, it's Facebook (and Twitter, and AIM)Google Wave is an upcoming product form Google, and has been described as a replacement for email. In the demo at Google I/O, real-time interaction was pervasive. Data in the format of emails, instant messages, tweets, and individual chat keystrokes were shared between multiple people at the same time. Participants were added in very granular ways - a whole chat, a paragraph of an email, an ongoing conversation starting from a specific point. Everyone proclaimed that Google Wave was The New Emailâ„¢, and would be used as such.<br /><br />Possibly, but Wave is far more powerful than that. At the heart of Wave is Jabber (XMPP), a technology that few people know the importance of. Some people might recognize it as the transport for Google Chat, but the real killer application of Jabber has been secure, real-time, in-house chat at financial and government organizations. Jabber is an efficient way for people to selectively share information with lots of people without needing a central server owned by an untrusted company.<br /><br />Jabber has not been a breakout success, likely because it cannot be used via a web browser, and instant messaging has a strong network effect. Google Wave exposes Jabber to web browsers via a custom Jabber server, AJAX, and the Google Web Toolkit. To combat the network effect, Google integrated their email service and then showed how two people using Wave would have an improved email experience. This embrace and extend approach will probably be the primary growth mechanism for Wave as a technology, and Google as featured API's for embedding Wave into existing web pages, and providing external services to Wave users.<br /><br />With a web browser interface, a Jabber back-end, and well documented extension API's, Wave is extremely useful. Write a robot for Twitter (which Google has already demonstrated as Twave), Flickr, and Blogger (also demonstrated by Google), and you recreate the core features of social media sites like Facebook. In fact, all that you would need to finish your proto-Facebook is some access permissions on the data (already part of Wave) and a mechanism for managing who has access to which data. Any data stored out on the external services (Twitter, Flickr, etc) will have their own permissions schemes, but data stored in Wave will be visible to people that you choose, just like Facebook. <br /><br />Your friends might not even have an account on the same Wave server, but due to Jabber federation, thats okay. One convenient reason for telling people that Wave is like email, is that people will add their server name when giving out their account, just like an email address: ynniv@ynniv.com, or ynniv@mac.com. GMail users are used to giving out their email address for all sorts of things: email, instant messaging, voice and video chat, document collaboration. <br /><br />If Wave turns out to be as important as I hope, people will stop differentiating their social networking account from their email account, and help break up the centralized control of personal information in the process. Back before the bubble, we thought that the Internet was going to be a decentralizing, democratizing force. Everyone on the Internet could send or receive data from anyone else. Email, and the web were products of this vision. Social networking and instant messaging has taken that away from us, placing all the control and everyone's private data in just a few hands. Jabber was created with that original decentralized vision, but never overcame the strong network effect in the instant messaging world. Google Wave could be the sugar to help the medicine down, finally bringing decentralization to the social web.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-3399175545962933535?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com4tag:blogger.com,1999:blog-7180892.post-77420456693853862472009-08-10T13:09:00.003-04:002009-08-10T14:21:19.780-04:00The Wave Way IS the Web WayAnil Dash at Lifehacker recently <a href="http://lifehacker.com/5332540/what-works-the-web-way-vs-the-wave-way">wrote</a> that Wave would face difficult adoption due to complex and rigid APIs. He says that the web has incremental upgrades, has a "weekend-sized" barrier to entry, has value independent of the network effect, and is easy to understand an explain (a duplicate of the second point, really). By comparison, he thinks that wave is big and complicated. It is my opinion that Wave applications will be similarly sized, similarly complex, and not require your friends to join at all.<br />He starts off by saying that Wave is composed of the following technologies:<br /><ul><li>Federation (XMPP)</li><br /><li>The robot protocol (JSONRPC)</li><br /><li>The gadget API (OpenSocial)</li><br /><li>The wave embed API (Javascript)</li><br /><li>The client-server protocol (As defined by GWT)</li></ul><br />and that "combining all of these pieces would just be the <i>starting point</i>" to development. This sounds to me like claiming that developing a Google Maps application involves working with NavTeq and writing tile rasterizers, or that making a website requires writing an OS, a web server, and a web browser. Here's what people will use to develop with Google Wave:<br /><ul><li>Make a wave view appear in your web page via the embed API (Javascript)</li></ul>and / or<ul><li>Make a view or interface in Wave using the gadget API (OpenSocial)</li></ul>and / or<ul><li>Provide outside data or services to Wave with the robot API (JSONRPC)</li></ul><br />All of them will have demo code that you can understand "in a weekend on your couch with a beer", or however else he thinks that people develop Web 2.0 apps. <br />Dash instead advocates a different API that does actually require you to host your own servers and federate over XML-RPC. The reasoning is that its easier for people to write their own server with XML-RPC than XMPP. XML-RPC is less efficient than Jabber (XMPP), so for intensive applications, should we be worried about performance? From the pushbutton page:<blockquote>Scaling issues? There will inevitably be some learning to do about how to scale the resource-intensive hub layer of a Pushbutton system. But because the hubs live on cloud systems that make enormous amounts of computing resources easily available, because the coders creating the reference implementations of the hub software have great experience making web-scale systems, and because it's relatively simple to introduce new hubs as needed, this will likely not be a gating factor for adoption of Pushbutton. Worry? No</blockquote><br />I think that last statement should read, "Worry? <b>YES</b>". Instead of using a properly written, efficient server, we should all run home brew, inefficient servers on huge clouds, because computing power is free! Google is going to release the Wave server open source, and its going to be really easy for you to run theirs on your hardware. Web programmers don't worry about writing web servers, and Wave programmers should not worry about writing a Wave server either.<br /><h2>Wave Is The Web</h2>Wave actually is the web, and people should stop comparing it to email. The Wave interface that Google demonstrated was a web page. It communicated with the browser via AJAX. A Wave interface element ("gadget") can be embedded into any web page, providing immediate functionality to anyone visiting that page, no account required. You will get additional functionality by creating an account, but there is still inherent value in receiving fresh data, and interacting with services in real-time, anonymously. There is no intrinsic network effect at play here. I predict that Wave is going to show up embedded in web pages everywhere in the form of live Twitter feeds and real-time collaborative features. No one has to know, they just keep doing what they're doing.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-7742045669385386247?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com0tag:blogger.com,1999:blog-7180892.post-87474757572035596972009-06-12T10:54:00.005-04:002009-06-12T10:59:55.083-04:00Improving Common Tag : Worse is BetterCommon Tag is an "open tagging format developed to make content more connected, discoverable and engaging" [<a href="http://commontag.org">commontag.org</a>]. It mixes RDFa into XHTML to add metadata to specify metadata for a content block, most importantly a link to a common database entry that iconifies the topic like the one word tags used in many places. This is an improvement over word tags, which can be non-descript or ambiguous: does "apple" refer a fruit, a computer company, a record label, or someone's name? Tags that are acryonyms may have no meaning to the user. RDF is commonly referred to as the Semantic Web, because it helps computers link concepts together. Everyone wants the Semantic Web, but somehow it never happens... maybe its because RDFa looks like this:<br /><pre><br /><body xmlns:ctag="http://commontag.org/ns#" rel="ctag:tagged"><br /> <span typeof="ctag:Tag" rel="ctag:means" <br /> resource="http://rdf.freebase.com/ns/en.u2"/><br /></body><br /></pre><br />This is a very explicit piece of data. Much of its content is XML support structure. The semantic knowledge contained in there is:<ul><li>the tag for this span is "en.u2" in freebase</li></ul><br />The structure contained in there (removing any content) is:<br /><ul><li>xmlns:ctag="http://commontag.org/ns#"</li><li>rel="ctag:tagged"</li><li>typeof="ctag:Tag"</li><li>rel="ctag:means"</li><li>resource="http://rdf..com/ns/"</li></ul><br />There are a lot of things that can break without actually removing any semantic meaning. If there were a typo anywhere in the structure above, your tag would be hopelessly borked - all that work (and bandwidth) for naught. More importantly, this format says that if I want people to understand my tags I have to embrace XML, and there are few things that I dislike as much as XML. Look at all the structure required because of XML, and the complicated tools that are required to manipulate XML!<br /><br />Lets take that content and put in something sexier: simple HTML. Over at Hacker News, <a href="http://news.ycombinator.com/item?id=653391">someone suggested</a><br /><ul><li><p ctag="wikipedia/The_Beatles">We're talking about The Beatles here</p></li></ul><br />I like this direction, but its lacking in a gruesome way: ctag is not a valid HTML attribute. Browsers may not like it, and they certainly won't be able to read it, so it isn't as clean as the RDFa. How about this:<br /><ul><li><p class="-ctag-wikipedia-The_Beatles">We're talking about The Beatles here</p></li></ul><br />This <b>is</b> valid HTML, browsers can operate on this content, and you can even style it! This is much better than the previous suggestion (which was quite good, and spurred me to write this article), but is it better than RDFa?<br /><br />Content:<br /><ul><li>the tag for this span is "The_Beatles" at Wikipedia</li></ul><br />Structure:<br /><ul><li>class="-ctag-"</li></ul><br />It is by far more concise than RDFa, but it has limitations - the tag content has to be valid inside a CSS class, which means alphanumeric, dash, and underline. There is additional flexibility if you use backslash, but this is uncommon in CSS classes and may not play nice everywhere. Most significantly, it doesn't include a link to wikipedia, only the name, and Semanic Web people really dislike that. I suspect that most people will link to Wikipedia, and if not, a search engine can figure out the most likely host. I mean, how "smart" are your tools when they can't deduce the meaning of wikipedia? If you're using an internal host or very specific database, you can always fall back to RDFa.<br /><br />Worse is better. The CSS tag format I propose is not as specific as RDFa, but it is easier to implement, harder to mess up, works with non-XHTML, and easy for humans to verify. These generally overlooked and undervalued qualities make adoption easier for people, which is in the end all that really matters.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-8747475757203559697?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com0tag:blogger.com,1999:blog-7180892.post-38122414545092179052009-05-04T11:53:00.003-04:002009-05-04T12:04:06.978-04:00The NYTimes Doesn't Understand Social NetworksIn <a href="http://www.nytimes.com/2009/05/03/business/03stream.html">"Tinker Away, Facebook Says"</a>, the Times summarizes Facebook's recent API expansion announcement with a comparison to a grocery store that "props open the front door and invites everyone to come in, take the merchandise free of charge, and then give it away themselves". They call this "the counterintuitive business wisdom infecting Silicon Valley these days", and proceed to imply that Facebook is doing this from the kindness of their heart.<br /><br />Let me suggest that Facebook is not a charity but a profit driven company, seeing the consumer primarily as an unavoidable obligation. An API expansion attracts additional users and (more importantly) encourages current users to not leave. A permissive developer API allows more people to participate in innovating their platform for little expense to Facebook. These companies not affiliated with Facebook take risks in developing features that users might want. If users like these features Facebook can incorporate them into the product, otherwise they will wither and die without tarnishing the Facebook brand.<br /><br />Developers see this as a way to capitalize (however meagerly) on the success of a powerful brand. Consumers see it as a new channel to cater to their needs. What would be better for both developers and consumers is a federated system with non-centralized intellectual property ownership, and thats exactly what Facebook intends to prevent. The most important thing that you should know concerning any Facebook API is that one thing remains the same: Facebook owns its customer's data, and no one, not even the customer, is allowed to export that data from them.<br /><br />To an extent, this bothers customers. Facebook recently changed their terms of service to say that Facebook owned the property rights all user entered content. Since this would have been catastrophic to artists and writers, there was a substantial backlash that resulted in Facebook rescinding this clause. They additionally apologized and claimed that this was never the intent. It seems unlikely that they would attempt to own all user content in this way, and I believe that they were honest in saying that this was a mistake. What the terms should have said is that Facebook claims ownership of the format and<br />context of data as it appears in Facebook. You own your posts, but the only way to use them outside of Facebook is to retype them by hand. You own your photographs, but maybe only if you have the originals. And under no circumstances can you export Facebook's bread and butter: the relationships that you have established.<br /><br />And so Facebook's goal is to make you just happy enough that you won't jump ship. They will claim as much ownership of content that they can for the sole purpose of making it hard to leave. They will make it difficult or impossible to remove content. They will sell as much of your identity as advertisers will buy without the government getting involved. These are what the investors of social networks talk about, and the only way that they can survive as free services.<br /><br />I am sad to see the Times miss what is blatantly obvious to me. Is it not the age old motto of investigative journalism to follow the money? It may be that our journalists have lost their curiosity. Or possibly that people have generally become consumers, and consumers see themselves not as powerful individuals but as reflected in their relationship to companies. With Social Networks, the money is in owning customer data as much as possible, and selling that data (via keyword advertising, or outright) to advertisers.<br /><br />If I impress anything on you, let it be that Facebook needs you more than you need Facebook. They need you to buy products so that they can sell advertising, because Facebook is not really a free service. Years from now, the person who owns your past (photographs, notes, messages with friends) will be the one paying for it now, and if you are a Facebook user, that person is not you. The best thing that you can do for your future self is to get involved and start owning your present identity. Data in a product like Wordpress, Blogger (but not Blogspot), or simple HTML on a web server has been paid in full. There is no question about who owns the rights to your past, or what you are allowed to do with it - it is simply yours. Certainly, Facebook has extra features that you cannot get from these truly free products, but comparable features won't exist in other products (free or consumer paid) until there is a demand for them.<br /><br />So get out there and encourage people to think about the past of the future, which might be tomorrow. Maybe even use Facebook to spread the word - but don't be surprised if they shut you down. After all, they are the ones footing the bill.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-3812241454509217905?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com1tag:blogger.com,1999:blog-7180892.post-40629157503663629932009-01-07T13:47:00.002-05:002009-01-07T13:52:57.074-05:00Firefox: An Acceptable Cross Platform GUI ToolkitA recent <a href="http://news.ycombinator.com/item?id=423119">Hacker News submission</a> asked what an "acceptable" cross-platform GUI toolkit would be. It discusses common complaints about the existing toolkits GTK+, Qt, Tk, and wxWidgets. It suggests creating a new toolkit with the qualities of being written in C, keeping it simple, LGPL licensing, easily skinnable for different OS's, binding for scripting languages, and be "simple and easy to use".<br /><br />Good news! I know of an existing GUI toolkit that fits this bill. Its written and extensible in C/C++, has liberal licensing, is very skinnable, has bindings for JavaScript, Python, Java, Perl, and Ruby, and lots of existing open source code that you can learn from. It even has a large, mixed platform install base, so you know that the bugs are minimal. Its based on an object brokering system called XPCOM, and an application framework called Mozilla, but often goes by the name Firefox. Given the popularity of web applications in the startup community, Firefox seems like an obvious option for desktop apps. Yet it hasn't caught on, and I'm not sure why. <br /><br />I recently worked on a desktop application for my day job. Given their limited interface goals and a game-oriented 3d engine, they had decided to write the interface directly in OpenGL. It was difficult to design interfaces like that, so I did a prototype integration with wxPython. That worked well enough to start real product development, during which I realized that wxPython had many of the "cross-platform toolkit" woes like needless internal complexity, difficulty in writing new components, and general ugliness.<br /><br />I spent some time contemplating how people wrote successful cross platform desktop applications, and the two things that struck me were (a) there aren't many, (b) except Firefox, (c) and this might underly the popularity of web applications. So I did a new prototype using Firefox with a XUL user interface. What I found was that Firefox worked out great as a standalone application with modular code design, but it was difficult to write good interfaces in XUL. XUL seems to be well tested only as far as the functionality in Firefox is concerned. We switched to Ext-JS, started pretending to be web app devs, and got down to implementing features.<br /><br />Writing a desktop app based on XULRunner has some interesting side-effects. For one, it makes retargeting your code as a web application or Firefox plugin really easy. It also lets you develop using a variety of hacker friendly languages. Like programming in lisp, it changes your perspective on how applications should be developed, and makes questions like "which UI toolkit is the best?" seem fundamentally flawed. In an age when Apple is rewriting their apps to run in a browser (and still look and feel like desktop apps), isn't writing a code against a desktop UI toolkit fighting the tide?<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-4062915750366362993?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com12tag:blogger.com,1999:blog-7180892.post-15482019646489522142008-12-10T13:48:00.007-05:002008-12-10T16:35:56.677-05:00JavaScript Performance<p>There are a lot of JavaScript performance benchmarks flying around out there. Some claim that Chrome shames the competition, others say that Firefox 3.1 is neck and neck with Chrome, others crown WebKit as the fastest of them all.</p><p>But what they never seem to do is compare them against other languages. One website, the <a href="http://shootout.alioth.debian.org">Computer Language Shootout</a> does, but it never seems to rate JavaScript particularly well. There are a few reasons for this. JavaScript has gotten a lot of attention recently and has thus been rapidly improving, and the CLS doesn't update very often. Tests are also written by different people, and folks who write JavaScript have never been the performance oriented crowd. Finally, the command line harness they use to execute JavaScript isn't representative of how users execute JavaScript.</p><p>So, what happens if we try to mitigate some of these? I picked a single test, rewrote it to run as a web page, and ran it using the latest version of browsers that can be considered stable. The results might surprise you.</p><br /><b>n = 10</b><table><tr><td>gcc</td><td>0.5s</td><td>1.0x</td></tr><tr><td>java6</td><td>0.7s</td><td>1.4x</td></tr><tr><td>java5</td><td>1.1s</td><td>2.2x</td></tr><tr><td>webkit</td><td>1.8s</td><td>3.6x</td></tr><tr><td>jsc</td><td>4.7s</td><td>9.4x</td></tr><tr><td>minefield</td><td>6.4s</td><td>12.8x</td></tr><tr><td>firefox</td><td>19.4s</td><td>38.8x</td></tr><tr><td>rhino</td><td>22.1s</td><td>44.2x</td></tr><tr><td>python</td><td>33.1s</td><td>66.2x</td></tr><tr><td>rhino</td><td>47.8s</td><td>95.6x</td></tr><br /><tr><td>ruby</td><td>58.3s</td><td>116.6x</td></tr><tr><td>webkit [parallels/xp]</td><td>1.0s</td><td></td></tr><tr><td>chrome [parallels/xp]</td><td>2.0s</td><td></td></tr></table><br /><b>n = 11</b><table><tr><td>gcc</td><td>5.1s</td><td>1.0x</td></tr><tr><td>java6</td><td>6.5s</td><td>1.3x</td></tr><tr><td>java5</td><td>12.6s</td><td>2.5x</td></tr><tr><td>webkit</td><td>23.4s</td><td>4.6x</td></tr><br /><tr><td>jsc</td><td>57.5s</td><td>11.3x</td></tr><tr><td>minefield</td><td>81.2s</td><td>15.9x</td></tr><tr><td>webkit[parallels/xp]</td><td>15.0s</td><td></td></tr><tr><td>chrome[parallels/xp]</td><td>29.0s</td><td></td></tr></table><br />Timings done on a Mac Book Pro 2.4 Gz Core 2 Duo / 4 GB RAM. Parallels/XP means running on Windows XP SP3 inside Parallels on the same computer.<br />WebKit is nightly build 39090.<br />WebKit [parallels/xp] is nightly build 39088.<br />jsc is built from svn rev 39090.<br />python is 2.5.1.<br />ruby is 1.8.6.<br /><br />So, interesting things:<ol><br /><li>The fastest JavaScript implementations are close to the speed of Java.</li><br /><li>The coming JavaScript implementations are substantially faster than Python or Ruby.</li><br /><li>WebKit is about twice as fast as jsc (the command line interpreter), so the Computer Language Shootout numbers will be inflated.</li><br /><li>Java 6 is substantially faster than Java 5.</li></ol><br />So, I expect to see more web services written in JavaScript. Why? The argument for Python or Ruby has been that they are much more productive than Java/C/C++/C# that the performance of these languages isn't important. I certainly agree with this. However, JavaScript has about the same level of language productivity, and now has an implementation thats 18 times as fast as python and 32 times as fast as ruby. And you can use the same language across the board for web apps.<br /><br />I also expect to start seeing desktop apps written in JavaScript. Why use a hacky python or ruby desktop app wrapper when you could use the best cross platform GUI kit there is? And, why bother with a local rails or django instance when you could do everything in a full MVC AJAX kit like <a href="http://www.sproutcore.com/">Sproutcore</a> or <a href="http://cappuccino.org/">Objective-J</a>?<br /><br />At the risk of giving away the secrets to my sauce, desktop JavaScript is going to change everything. There are already a couple of frameworks out there (<a href="http://www.aptana.com/jaxer">Jaxer</a>, <a href="http://titaniumapp.com/">Titanium</a>), but I think that there will be many more to come.<br /><br />(PS: Lisp FTW!)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-1548201964648952214?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com3tag:blogger.com,1999:blog-7180892.post-52956090313333917802008-08-04T16:01:00.005-04:002008-08-20T10:14:50.193-04:00Java Static Class Object Fields and Weak ReferencesIn browsing the Mozilla Rhino code, I stumbled upon a garbage collection problem that is truly bizarre. Rhino uses thread local storage for the current Context object. TLS uses a weak dictionary, so it is important that there is no strong circular reference to the key. Years ago (CVS r 1.234), the rhino class Context used to have a static reference to the thread local storage object. This was a problem because even when an array is empty, it has a reference to the class of objects it can hold. Thus, the TLS key (the ThreadLocal object itself) is referenced by the Context class, which is referenced by the (empty) Context[].class, which is a value in the TLS map! This circular dependency caused rhino to leak memory whenever a thread that used JavaScript completed. The solution to this was to change the value to an Object[]. When there are Context objects in the array, the reference still exists, however once the array is empty, there is no longer a reference and the TLS weakref dictionary will cut that key/value pair loose.<br /><br />The <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=281067#c5">comment</a>, <a href="http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/js/rhino/src/org/mozilla/javascript/Context.java&rev=1.234&root=/cvsroot">original</a>, and <a href="http://bonsai.mozilla.org/cvsview2.cgi?diff_mode=context&whitespace_mode=show&file=Context.java&branch=&root=/cvsroot&subdir=mozilla/js/rhino/src/org/mozilla/javascript&command=DIFF_FRAMESET&rev1=1.234&rev2=1.235">fix</a>.<br /><br />Wow, thats a lot of detail. Here's the summary: if you have static class fields, make sure they don't eventually point to the key of a weak dictionary. If find this unavoidable, you can break the loop by storing a value as an Object and casting any usage of it. You will still have to set this variable to null in order for GC to happen.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-5295609031333391780?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com1tag:blogger.com,1999:blog-7180892.post-38101136753753500452008-06-17T16:00:00.003-04:002008-06-17T16:08:53.440-04:00Firefox 3 is the Ugliest Browser YetIt's a bit trite for a blog post, but seriously, Firefox 2 looked way better than 3 on Windows. If you customize the toolbar (via right click) to use "small icons", its marginally better, but people actually spent time working on the design of this UI, so I shouldn't have to fix it for them. Everyone would have been much better off with the third party Firefox 1.x theme <a href="http://members.shaw.ca/lucx/">Charamel</a>. <br /><br />I haven't used the Mac version yet, but it appears to have the same heinous icon problems. Look at Safari and try again please.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-3810113675375350045?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com4tag:blogger.com,1999:blog-7180892.post-50425369198446919762008-05-29T17:39:00.003-04:002008-05-29T17:43:38.057-04:00XPCOM nsISupports Proxy Crashes<p>In building an application based on XULRunner, we've uncovered a number of bugs in the Mozilla codebase. One that really bothers me is the implementation of asynchronous object proxies. First, here's a brief overview of nsISupports proxies.</p><h3>XPCOM</h3><p>XPCOM is a generalized system for calling methods on objects. These objects can be implemented in C++, JavaScript, or Python (using a 3rd party library called PyXPCOM). XPCOM is the mechanism for defining Firefox extensions and writing applications that use XULRunner.</p><h3>Working With Threads</h3><p>One handy feature of XPCOM is the ability to have your method call execute in another thread. One example of this is a background thread that wants to do something to the UI, such as update a progress bar. Similarly to Java, Cocoa, and some Windows programming, UI access in Mozilla has to happen on the main thread. Another common situation is sending commands to a worker thread from the main thread. In both cases, we have a simple command to issue, but a direct method call is the wrong way to accomplish this.</p><p>So XPCOM has a mechanism for defining proxy objects that look identical to the original object, but executes your request in another thread. There are two flavors of this, synchronous (PROXY_SYNC) which will block your thread, and asynchronous (PROXY_ASYNC) which queues a request in the other thread and continues on. One implication of asynchronous dispatch is that there will be no return value, since you don't wait for the method to finish.</p><h3>The Async Warning</h3><p>The page <a href="http://developer.mozilla.org/en/docs/nsISupports_proxies">describing proxies</a> contains a very specific warning: if you pass variables by reference and those variables are on the stack, bad things will happen. It turns out that this is a bit understated, and doesn't convey some of the subtleties, so I will try to clarify.</p><h3>Asynchronous XPCOM Proxies Are Fundamentally Broken</h3><p>Using them will likely cause you nothing but pain. This pain will be in the form of totally random crashes, with meaningless (but seemingly useful) stack traces. You will ask people for help, and they will have no idea where to start. Unless you are intimately familiar with the XPCOM message dispatch and QueryInterface source code, PROXY_ASYNC should be a keyword which in your mind expands to Bad Bad Evil Crashing.</p><p>Why? Well, it turns out that return values are implemented as "out" parameters, which is a fancy way of saying references on the stack, which you may have recently learned is something that you should never mix with PROXY_ASYNC. So, if you were to call any method on an asynchronous proxy that <b>could</b> return a value, it will corrupt memory. There's actually no uncertainty here, XPCOM will write over some random memory address which is probably on some poor thread's stack.</p><h3>The Workaround</h3><p>"Ah!" you say, I can use PROXY_ASYNC and still sleep at night as long as I only call methods that return "void". Well, you might sleep a night or two, but soon you'll realize that bad things are still happening. The reality is, there is no workaround (sorry!). Even if you avoid methods with "out" or "in out" parameters, and ones that return values, Mozilla will still punish you by calling exactly these sorts of methods internally. If you're using a scripting language (generally JavaScript), things are even worse for you, because it makes a bunch more of these calls to set up your script object.</p><h3>The Documentation</h3><p>This bug is currently documented in <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=382631">buzilla</a>. Unfortunately, there seems to be no workable solution to the situation. On top of that, the people responsible for the code base don't know why anyone would even want to use asynchronous dispatch. So, that doesn't sound good. The only real fix is to find a way to remove any mention of PROXY_ASYNC from your code. Depending on your application, this may be an easy fix. So far for us, this has not been the case. (Mozilla was never intended to do what we're doing)</p><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-5042536919844691976?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com1tag:blogger.com,1999:blog-7180892.post-34722179585980733802008-01-04T12:44:00.000-05:002008-01-04T12:45:11.363-05:00FYI: JavaScript is LispJust a heads up in case anyone hasn't figured this out yet.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-3472217958598073380?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com2tag:blogger.com,1999:blog-7180892.post-23809360972004818482007-11-17T17:05:00.000-05:002007-11-17T18:15:57.131-05:00Some Things Never ChangeI just finished watching a <a href="http://video.google.com/videoplay?docid=8612534856516244040">half hour video on computer graphics</a>. It covers the basics of 2D and 3D graphics and basic animation in a way that the average person can understand, but with enough detail to provide basic understanding.<br /><br />The information itself isn't particularly blog-worthy; there are plenty of resources for learning about computer graphics. There are two reasons that this is interesting to me. First, television seems to have a dwindling amount of educational content. I remember watching <a href="http://en.wikipedia.org/wiki/3-2-1_Contact">3-2-1 Contact</a>, <a href="http://en.wikipedia.org/wiki/Mr_wizard">Mr. Wizard</a>, and <a href="http://en.wikipedia.org/wiki/NOVA">NOVA</a> as a kid. The programs <a href="http://en.wikipedia.org/wiki/List_of_programs_broadcast_by_Public_Broadcasting_Service#PBS_Kids">currently on PBS</a> don't seem to show the same interest in science, substituting fantasy, social interaction, and ironically, reading. This may be related to my perception that they are also oriented to a younger audience, but I can't help but feel that kids should be learning these things <i>instead</i> of watching TV, not <i>by</i> watching it.<br /><br />Second, this video is from a program called <a href="http://en.wikipedia.org/wiki/Sol_Garfunkel">For All Practical Purposes</a> and was filmed in the late '80s. The computer systems used are from <a href="http://en.wikipedia.org/wiki/Symbolics">Symbolics</a>, a company that made their own hardware and operating system written entirely in lisp. This idea is similar to the current <a href="http://en.wikipedia.org/wiki/Squeak">Squeak</a> project, which is written in smalltalk instead of lisp. The video is old, the computers are old, the technology is old, and yet, it isn't particularly different from the basic concepts that I learned at Georgia Tech a few years ago. The biggest changes have been in the hardware that allows us to animate using physical simulation, and have realtime rendering.<br /><br />Modern computing doesn't seem that different from 80's computing, except that everything is <a href="http://en.wikipedia.org/wiki/Image:Moore_Law_diagram_%282004%29.png">faster</a>, <a href="http://en.wikipedia.org/wiki/Image:Hard_drive_capacity_over_time.png">larger</a>, <a href="http://en.wikipedia.org/wiki/Wirth%27s_Law">less efficient</a>, <a href="http://en.wikipedia.org/wiki/Mac_OSX">prettier</a>, <a href="http://en.wikipedia.org/wiki/Broadband_access">more connected</a>, and cheaper (much, MUCH <a href="http://en.wikipedia.org/wiki/Macintosh_II">cheaper</a>). Just today I learned of a London startup that is writing games that would have run on a 1.2 MHZ <a href="http://en.wikipedia.org/wiki/Atari_2600">Atari 2600</a> in Macromedia Flash - which uses most of my 1,200 MHZ CPU. But ultimately, its all the same. In my eyes, this makes a stronger case for the idea of Computer Science, because if you ever really learned how computers work, you might understand them forever.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-2380936097200481848?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com1tag:blogger.com,1999:blog-7180892.post-2816094522561178182007-11-16T15:40:00.000-05:002007-11-17T18:32:13.808-05:00parent and __parent__ and __proto__, Oh My!Sometimes Javascript development can be rather scary. Unlike C or Java, which have well known standards, Javascript (like HTML) is often described by what works on one or more of the current implementations. On top of that, few people see Javascript as a legitimate language, and a vast number of novices are out there writing code that would make your skin crawl. <br /><br />In short, if you are a knowledgeable programmer, quickly picking up Javascript is much harder than it should be. The syntax is C-like (generally regarded as a plus), the functions are first class (almost universally positive), and the object system is prototype based (at least smalltalk people like this). Soon there will be a common <a href="http://www.mozilla.org/projects/tamarin/">just-in-time compiler</a>, and some people have suggested that it might <a href="http://steve-yegge.blogspot.com/2007/02/next-big-language.html">displace currently popular languages</a>. Personally, I wish that it ditched the prototype object system for a Python-like one, and had lisp-like macros, and just-in-time compilation can't come soon enough.<br /><br />When a question about a Javascript feature like "x.__proto__" arises, it can be difficult to answer. In my experience so far, the best places to look are <a href="http://developer.mozilla.org/en/docs/JavaScript">MDC</a> and the IRC channel listed on the page.<br /><br /><span style="font-weight:bold;">What is __proto__?</span><br /><br />When you make a Javascript "class" (ie constructing function), you can specify a prototype:<pre>function Example() {<br /> this.foo = 1;<br />}<br /><br />Example.prototype = {<br /> sample : function memberFunction() {},<br />}</pre>This attribute is copied onto constructed objects as <code>__proto__</code>, like so:<br /><pre>ex = new Example();<br />if (ex.__proto__ == Example.prototype) print("same!");</pre><code>__proto__</code> is a special attribute that is searched if an attribute is not found on the object, it is analogous to a parent class.<br /><br /><code>parent</code> and <code>__parent__</code> are substantially different. <code>parent</code> comes from the DOM, and is the tag that encloses the current tag. <code>__parent__</code> is a special variable that is a reference to the scope that created the object. Like <code>__proto__</code> is checked when a variable lookup fails. Where <code>__proto__</code> is checked when the failure looks like <code>this.missing_attribute</code>, <code>__parent__</code> is checked when the failure is in the form <code>missing_global</code>. According to <a href="http://www.mozilla.org/rhino/overview.html">Mozilla</a>, <code>__parent__</code> and <code>__proto__</code> have been deprecated, which is unfortunate since they can be quite useful.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-281609452256117818?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com3tag:blogger.com,1999:blog-7180892.post-1605405855890674852007-11-05T23:41:00.000-05:002007-11-06T17:08:23.881-05:00Debugging PythonAbove all else, my greatest annoyance with python is the lack of good documentation and defaults. I bet that there is a group somewhere that knows everything there is to know about python... I beg of them, write that knowledge down!<br /><br />Here's an example. I want to debug a faulty program:<br /><pre>x = [1,2,3]<br /><br />def foo():<br />sum = 0<br />for i in range(4):<br /> sum += x[i]<br /><br />import pdb<br />pdb.run("foo()")<br /></pre>I'm then prompted by pdb: (> replaced with ])<br /><pre>] [string](1)?()<br />(Pdb)<br /></pre>Good, now I'm in the debugger. To get things started, I type "c", return, and get:<br /><pre>Traceback (most recent call last):<br />File "[stdin]", line 1, in ?<br />File "c:\Python24\lib\pdb.py", line 996, in run<br />Pdb().run(statement, globals, locals)<br />File "c:\Python24\lib\bdb.py", line 366, in run<br />exec cmd in globals, locals<br />File "[string]", line 1, in ?<br />File "[stdin]", line 4, in foo<br />IndexError: list index out of range<br />]]]<br /></pre>You should notice that the last line is the standard python prompt (with the aforementioned character replacement to make blogger behave), not the debugger. I have finished my debugging session due to an error. Gee, it sure would have been nice to debug that, since my state is now lost! I was already using the debugger to run this code, why did it not catch this exception?<br /><br />It turns out that exception handling is done by sys.excepthook, and pdb.run doesn't set the excepthook. Some searching turned up two options. The first is simple but crude - add the following to site-packages/sitecustomize.py:<br /><br /><a href="http://mail.python.org/pipermail/python-list/2001-April/079168.html">Thomas Heller</a><br /><pre>import pdb, sys, traceback<br />def info(type, value, tb):<br /> traceback.print_exception(type, value, tb)<br /> pdb.pm()<br />sys.excepthook = info</pre>The second is more sophisticated, and checks for interactive mode:<br /><br /><a href="http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65287">ActiveState Python Cookbook</a>.<br /><pre># code snippet, to be included in 'sitecustomize.py'<br />import sys<br /><br />def info(type, value, tb):<br /> if hasattr(sys, 'ps1') or not sys.stderr.isatty():<br /> # we are in interactive mode or we don't have a tty-like<br /> # device, so we call the default hook<br /> sys.__excepthook__(type, value, tb)<br /> else:<br /> import traceback, pdb<br /> # we are NOT in interactive mode, print the exception...<br /> traceback.print_exception(type, value, tb)<br /> print<br /> # ...then start the debugger in post-mortem mode.<br /> pdb.pm()<br /><br />sys.excepthook = info</pre>Here's what I'm currently running:<br /><pre># code snippet, to be included in 'sitecustomize.py'<br />import sys<br /><br />def info(type, value, tb):<br /> if (#hasattr(sys, "ps1") or<br /> not sys.stderr.isatty() or <br /> not sys.stdin.isatty()):<br /> # stdin or stderr is redirected, just do the normal thing<br /> original_hook(type, value, tb)<br /> else:<br /> # a terminal is attached and stderr is not redirected, debug <br /> import traceback, pdb<br /> traceback.print_exception(type, value, tb)<br /> print<br /> pdb.pm()<br /> #traceback.print_stack()<br /><br />original_hook = sys.excepthook<br />if sys.excepthook == sys.__excepthook__:<br /> # if someone already patched excepthook, let them win<br /> sys.excepthook = info</pre>The original ActiveState script doesn't debug if you are running in interactive mode. To me, this makes no sense at all - thats a case where I specifically want to debug. <br /><br />Alas, if the debugger shows an obvious error that could be fixed, python exceptions cannot be resumed. The code listed here will let you see the stack and move around in it, but your program is no longer running and can never finish where it left off. Allowing this requires call/cc, and as far as I can tell there is no plan to ever support that in python.<br /><br />If this was helpful, or should be changed, let me know. I've just started using it myself.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-160540585589067485?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com3tag:blogger.com,1999:blog-7180892.post-86888950726524618112007-08-12T09:37:00.001-04:002009-05-19T12:06:34.527-04:00Closures in PythonA closure is data attached to code (pretty simple, eh?). I use them for:<br /><ul><li>Replacing hard coded constants</li><li>Eleminating globals</li><li>Providing consistent function signatures</li><li>Implementing Object Orientation</li></ul>(Isn't it funny that people rarely tell you what closures are good for?)<br />Here is a closure in python:<br /><pre>def makeInc(x):<br /> def inc(y):<br /> # x is "closed" in the definition of inc<br /> return y + x<br /><br /> return inc<br /><br />inc5 = makeInc(5)<br />inc10 = makeInc(10)<br /><br />inc5 (5) # returns 10<br />inc10(5) # returns 15</pre>Closures in python are created by function calls. Here, the call to makeInc creates a binding for <b>x</b> that is referenced inside the function inc. Each call to makeInc creates a new instance of this function, but each instance has a link to a different binding of <b>x</b>. The example shows the closure of <b>x</b> being used to eliminate either a global or a constant, depending on the nature of <b>x</b>.<br /><pre>import time<br />keepRunning = True<br />updates = []<br />def runLoop():<br /> while (keepRunning):<br /> for u in updates:<br /> u()<br /><br />class foo:<br /> def __init__(self, x = 0):<br /> self.x = x<br /><br /> def update(self):<br /> print self.x<br /> self.x += 1<br /><br />f = foo()<br />g = foo(2)<br /><br />updates.extend([f.update, g.update])</pre>In python, all methods (but not functions) are closures ... sort of. The method definition <b>foo.update</b> closes the class foo. The value of <b>g.update</b> is a closure that stores the value of <b>g</b> and passes that as the first argument of <b>foo.self</b>, hence the first argument of a method in python is <i>self</i>. Details aside, it is important to note that the designers of python have gone out of their way so that you can pass <b>g.update</b> by itself to another function and have it continue to work correctly.<br /><br /><span style="font-size:140%;">Caveats</span><br /><br />In some languages, the variable bindings contained in a closure behave just like any other variables. Alas, in python they are read-only. This is similar to Java, and has the same solution: closing container objects. Closure of a dictionary or array won't let you assign a new dictionary or array, but will let you change the contents of the container. This is a common use pattern - every time you set a variable on <i>self</i>, you are changing the contents of a closed dictionary.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-8688895072652461811?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com7tag:blogger.com,1999:blog-7180892.post-12823269420445805042007-08-03T01:39:00.000-04:002007-08-03T01:40:21.705-04:00A Visualization of VisualizationsYou really need to see this one first hand, so <a href="http://www.smashingmagazine.com/2007/08/02/data-visualization-modern-approaches/">check it out</a>.<div><br /></div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-1282326942044580504?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com3tag:blogger.com,1999:blog-7180892.post-6248067908338626362007-07-06T19:13:00.000-04:002007-07-06T21:38:30.642-04:00Working with Python/C<p>Some very short notes for working with python from C. I'm using Python 2.4.x, YMMV.</p><p><br />Getting the current error state inside your debugger:<br /></p><pre>_PyThreadState_Current->curexc_type</pre><p></p><p><br />Instantiating a "New Style" class (a subclass of "object"):</p><pre>PyObject* args = PyTuple_New(0);<br />PyObject* dict = PyDict_New();<br /><br />// works only with new style classes (subclasses of "object")<br />PyObject* result = PyObject_Call(classObject, args, dict);</pre><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-624806790833862636?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com0tag:blogger.com,1999:blog-7180892.post-34785213457970108582007-06-15T12:54:00.000-04:002007-06-15T13:17:39.383-04:00Open Network Sockets on Mac OS X<p>This is one of those things that constantly annoys me. On Linux, <code>netstat</code> tells me the list of currently active network connections, including (often most importantly) listening connections. Just knowing that something is running on port 8080 tips me off that I probably have an Apache or Java EJB process running (or maybe a <a href="http://wiki.squeak.org/swiki/">swiki</a>) - a trip to localhost:8080 will answer my question. But what if this port isn't an HTTP server, and doesn't speak when connected to? Now you have a dilemma - there's no way of knowing what has this port open.<br /></p><p>Fortunately this is a solved problem on Linux. <code>netstat</code> has some options that tell you the PID of the process with the port open. From here you can use <code>ps</code> to find the name of the process, its path, the user who started it, etc. BTW, you need to run netstat as root to see other people's PID's and those of services.<br /></p><p>Well, you know what comes next - <code>netstat</code> on the Mac doesn't show PID's! WTF! Speaking of commands that Mac OS X doesn't have, <code>fuser</code> is missing as well. <code>fuser</code> on Linux tells you which processes have a specific file open - very useful if you're cleaning up files and one has is locked. Well, Mac OS X (BSD really) doesn't have <code>fuser</code>... but it does have a command called <code>lsof</code>. <code>lsof</code> isn't quite as user friendly as <code>fuser</code>. It only has one mode, which is to list every open file that is visible to you (this is a subtle hint that you should run it as root to see more files). This means that <code>fuser <filename></code> roughly translates to <code>lsof | grep <filename></code>. Very useful for finding that stray service that has outlived its welcome and is holding files hostage.</p><p>Still, the problem at hand is finding the PID of network sockets. It turns out that in POSIX, network sockets are pretty much the same as files. This means that they show up in <code>lsof</code> if you ask nicely. And since <code>lsof</code> shows PID's (it even gets fancy and shows the process name), it turns out to be the solution. So, here's the money shot:</p><p><code>sudo lsof -i -P</code></p><p>This produces something that looks like:</p><pre><br />COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME<br />launchd 1 root 9u IPv4 0x01d96e10 0t0 UDP *:137<br />launchd 1 root 10u IPv4 0x020dbe8c 0t0 TCP *:139 (LISTEN)<br />launchd 1 root 11u IPv4 0x020dbb38 0t0 TCP *:445 (LISTEN)<br />launchd 1 root 12u IPv6 0x01d99c50 0t0 TCP *:22 (LISTEN)<br />launchd 1 root 13u IPv4 0x020db7e4 0t0 TCP *:22 (LISTEN)<br />mDNSRespo 43 root 7u IPv4 0x01d96ad0 0t0 UDP *:5353<br />mDNSRespo 43 root 8u IPv6 0x01d96a00 0t0 UDP *:5353<br />mDNSRespo 43 root 9u IPv4 0x03379d40 0t0 UDP *:5353<br />mDNSRespo 43 root 13u IPv4 0x03379860 0t0 UDP 192.168.1.100:53891<br />mDNSRespo 43 root 14u IPv4 0x032f82a4 0t0 TCP *:* (CLOSED)<br />mDNSRespo 43 root 15u IPv4 0x02edb554 0t0 TCP 192.168.1.100:5000 (LISTEN)<br />mDNSRespo 43 root 16u IPv4 0x020da098 0t0 TCP *:* (CLOSED)<br /><br />... (lots more here)<br /></pre><p>If you don't have root access, you can still use <code>lsof</code>, but you won't see the plethora of system services and other users processes.</p><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-3478521345797010858?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com6tag:blogger.com,1999:blog-7180892.post-1163742265026572662007-05-29T12:43:00.000-04:002007-05-29T14:42:51.762-04:00Python Techniques You've Been Googling ForI've been doing it, I bet you have too. Here's some topics that have come up lately, I'll be writing about each of them in time. Since you probably got here via Google, add a comment for the subject that got you here.<br /><br />Closures<br />Debugging<br />Creating functions<br />Metaclasses<br />Performance Optimization<br />Objects in depth<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-116374226502657266?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com1tag:blogger.com,1999:blog-7180892.post-52408948715265290622007-03-17T13:45:00.001-04:002007-03-17T13:45:16.776-04:00asdf-install on openMCLA Note: This has been sitting in my drafts folder for 10 months. I'm<br>posting it unedited because its better out then in, right?<p>---<p>This should be pretty trivial... after all, its included with the<br>latest openMCL, right? (I'm using the latest CVS)<p>Well, I've been fighting to get it working for almost a week, most<br>likely because I tend to ignore files named README. Having some<br>experience with CPAN and rubygems, it should really be possible to get<br>packages from a package repository without even knowing the language<br>that you are using. Why? Because newbies want to see results before<br>they have to invest time in learning new things. In the worst case,<br>an end user doesn't know, or even want to know, how to program at all.<br> Package repositories should really just work.<p>Here are my notes so far:<p>You need to compile asdf-install. To do so, you should be in the<br>directory containing asdf-install.asd . Then run:<br>(asdf:operate 'asdf:compile-op :asdf-install)<br>(asdf:operate 'asdf:load-op :asdf-install)<br>Now, make your asdf registry:<p>mkdir /usr/local/share/ccl/asdf-registry<br>cp asdf-<p>You need to create ~/openmcl-init.lisp with:<br> (require 'asdf)<br>(setf asdf:*central-registry*<br> '(*default-pathname-defaults*<br> #p"/usr/local/asdf-install/site-systems/" ; where<br>asdf-install puts things<br> #p"/usr/local/share/ccl/asdf-registry/" ; where i put<br>asdf-install.asd<br> ))<br>(asdf:operate 'asdf:load-op :asdf-install)<p>;;; its really best to install GPG<br>(setf asdf-install:*VERIFY-GPG-SIGNATURES* nil)<p>The central registry setup is important - maybe<br>asdf-install/site-systems should be the same folder as the asdf<br>registry... I dunno. What I do know is that they both need to be in<br>the asdf search path (ie asdf:*central-registry*) or you won't get<br>very far.<p>I hope that I'm misguided, this is all a bad dream, and if I only<br>followed some magic instructions, this would be trivial. For now,<br>this works for me, and thats all that really matters.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-5240894871526529062?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com0tag:blogger.com,1999:blog-7180892.post-1165200351844756592006-12-03T21:37:00.000-05:002006-12-03T22:40:53.800-05:00Old And In The Way (stale fasl's)SBCL 1.0 has been released, and as expected, it breaks some things on my Powerbook. One specific problem is stale "fasl" files. Honestly, I don't even know what they are... I suspect them to be some kind of optimized bytecode used by ASDF to improve loading speed, but really (as long as they work) I don't care what they are.<br /><p>SBCL will stutter something like this:<br /><code><br />debugger invoked on a SB-FASL::INVALID-FASL-VERSION:<br /> #<SB-SYS:FD-STREAM for "file /usr/local/lib/sbcl/site/rt-20040621/rt.fasl" {11A024F9}><br /> is in native code fasl file format version 70, but this version of SBCL uses<br /> format version 71.<br /></code><br />Searching cliki revealed that these are <a href="http://www.cliki.net/asdf">intermediate files</a> that can simply be deleted. They suggest that deleting files by hand is tedious and have a nice code snippit that should make things magically refresh in the future... I'm a little more pragmatic:<br /><code><br />find /usr/local/lib/sbcl -name '*.fasl' -exec rm {} \;<br /></code><br />Works fine afterwards.<br /><br /><b>Update:</b> I was a little quick to declare victory. After executing this, i was unable to (require 'asdf), since the fasl for asdf had been deleted. I reinstalled (executed sbcl-1.0/install.sh) and the problem is fixed. This seems to invalidate my previous suggestion... Maybe I should have filtered the search to only include files older than a certain date. The cliki code snippit is probably the better way to go for now.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/7180892-116520035184475659?l=ynniv.com%2Fblog' alt='' /></div>ynnivhttp://www.blogger.com/profile/08240692067085029643noreply@blogger.com0