After posting earlier about performance problems with Dropbox, I got an email from Arash Ferdowsi, the Founder/CTO of the company, saying there is a new version in the final stages of testing. He suggested it might help with some of the problems.

He wrote: "I recommend trying the latest release candidate of dropbox. it's been incubating for the last 9 months and we're planning to release it in the next 2-3 weeks. it's much better at handling large file sets and uses much less cpu/memory. additionally, it's a lot more aggressive about pruning the dropbox cache (the build you're on only clears it when the dropbox process reboots while the new build clears it every 3 days)."

http://forums.dropbox.com/topic.php?id=27419&replies;=146

I asked if I could install the software on just some of the machines here, and he said yes, with a caveat. "0.8 now supports syncing of the executable bit and some other xattr data. if you're running a mix of 0.7 and 0.8 clients, this metadata won't sync properly."

I'm installing the new release on the server that's suffering the most, and on one or two other machines. I will of course report the results.

Update: I've tried installing the new software on two machines and both installs have frozen at this point. I'm going to leave the installer running on this machine for a while and see what happens.

Update: A couple of hours later, I was able to get it to install on one of the machines by first uninstalling then re-installing the app. This meant relinking the account, a slow process, but there was an upside -- I stumbled across the new Selective Synch function. This is going to make a big difference. It allows me to push content out through Dropbox that I might only need on one or two other machines, without adding overhead for the others. Nice!

Update: Why does Dropbox download so slowly? 5K/sec. I have no bandwidth constraints in the prefs. See the screen shot.

 11/27/2010; 4:12:11 PM.   .

I've spent the last week wrestling with my servers, dividing up what they do, factoring things so that maybe I can get the performance where it needs to be. It's one of the reasons I was exploring the possibility of using S3 to store my static sites (I gave up, couldn't make it work).

So now things are running in the new configuration, or I should say -- limping along in the new configuration. There is always lots of breakage when I do this kind of surgery. While doing it I wish I could integrate DNS, generating Apache config files and serving into one master outline, where I make a change and the new configuration automatically percolates where it needs to. That, and it would be great to have one server with infinite CPU, disk and net bandwidth, so I'd never have to logically split a server across multiple machines.

Maybe I'll get all this in time for next Christmas!

Anyway...

I think the mistake I made in the last half of 2010 is that I got too Dropbox-happy.

I built too many systems on it, without understanding that it could be brought to its knees, begging for less to do, all the while causing every other application on the server to be starved for CPU cycles.

When I saw how much work Dropbox was doing just to keep current in a relatively stable folder, I realized I needed to lower my dependence on it. It really wasn't doing that much for me, but it was taking up a very large portion of the CPU cycles on every one of my machines. (Only a few small text files were changing every few minutes, the really big stuff came once a night when the servers backed up).

Also it uses a huge amount of disk space. The cache is enormous. I figure it uses at least 1.5 times the size of the folder.

Now, Dropbox is a miracle product. Last year I called it an "idea factory." It gets you thinking about great things could be with a simpler architecture, and closer to the ideal of one big server without boundaries. It's a very easy-to-understand queueing system, and when you add XML or JSON to the mix, you get the magic that comes at the intersection of simplicity and power. But that's the virtuality of it, the reality is that it uses a lot of resources, so you have to use it carefully.

I'm interested in finding out what other people are learning about Dropbox.

 11/27/2010; 4:25:24 AM.   .

Interesting piece in the Telegraph quoting new Twitter CEO Dick Costolo saying that Twitter doesn't have a long-term vision, but it's his job to come up with one, and we should expect to hear something there shortly.

I think what he did here was admirable because it's so obviously true. It invites people like yours truly to pontificate publicly on what we think their vision should be. Maybe someone somewhere has an idea, and cares enough to share it with the rest of us. Maybe there will be something worth listening to in all the noise?

So what do I think Twitter's vision should be?

It seems, no matter how you slice it, they have the same two choices that all tech companies do:

1. An ad-based business, where the users get access for free, and they sell the attention of and aggregate data about the users.

2. Let the users run their own ads, if they want, and charge them to use the system. A freemium model, where the base product is free, but advanced features that help you sell stuff through Twitter, cost money. (Users might even come up with new ways of advertising, given a chance to be creative.)

In the first model, it's inevitable that not only will Twitter force developers off their platform (as they already are doing) but the same will eventualy happen for news organizations that use Twitter as their notification system. As with developers there's a naivete among news orgs that Twitter is somehow a public utility. This is not how Twitter views itself, nor is it in reality what it is (it's a privately held corporation).

In the first model they basically compete with everyone who uses their service as part of their business. In the second model, they compete with almost no one, and simply make money off the growth of the activity.

I think they've gotten so large so quickly that #2 is the only viable approach. It's the same advice I gave Microsoft before they hit the wall with the Internet, and the same advice I gave Google as they were hitting the Facebook wall. Stop hiring outside the basic area you already occupy, and shrink that if possible, and get ready to take 50 percent of all the growth, and share the rest with others. It clearly would have worked better for Microsoft, and not so clearly (yet) for Google. But it's a predictable cycle that Twitter is advancing through very quickly because they're growing so fast, and because unlike Microsoft and Google, they never had a golden age when they were technically best at what they did. They've always been ragtag, and I'm pretty sure they always will. It's in their DNA. And besides, companies as they grow never get more competent, they always get less.

But Twitter has been advancing fast into model #1 -- advertising and aggregation of user data. The purpose of advertising is to turn cash into attention. Companies will be diseconomic about this, for a while -- but long-term they can't afford to. A company that bases its business on buying attention must get more value back than the money they spend.

People seek out commercial information on the Internet all the time. They look for it. Think about your own habits. Last time you bought something, how much time did you spend finding out about it on the net. I won't even go out to eat at a restaurant that looks good from the street, that was recommended by someone I like, without looking it up on Yelp and seeing what other people thought. It's amazing to me how unhelpful most vendors' sites are in my buying decision.

Ads almost never influence my buying decision.

Companies that spend a lot on advertising would be well-advised to instead spend the money on making their websites more helpful for people wanting to buy their products. Long-term that works, imho. Unfortunately for Twitter, their service isn't very good for providing info about products people may want to buy. I can't even keep track of what cities my friends are in with Twitter, the pulse is so rapid and the flow is so flat (which is good thing for news, btw).

Further, more and more product development is going to shift to the net. That manufacturing companies are going to become fullfillment houses for communities of users who know what they want, and contract to have it made. So the flow of information the other way, the opposite of advertising, is very important. Not just the aggregated statistical info where we're all unidentifiable hamsters turning wheels around, the other kind -- intelligence, smarts, even brilliance and vision, that comes from not being immersed in the very limited vision of Silicon Valley which boils down to "we know what's right." What's great about what Costolo said is that he admits they don't know what's right. Hold on to that, and never let it go. The wisdom about your product is not in SOMA, it's "out there" -- the rest of the world that SV works so hard to abstract and simplify. It's not simple, it's rich and very very different from what you think it is.

What would be really smart is for him to say "Hold on, I'm going to take a few months here and really start listening to what the users think about this product, without any expectations." And really try to understand. Tht would do more long-term good for Twitter than what's more likely to happen -- a series of off-sites and talking with tech industry insiders, and more viral ad-driven money schemes.

Bottom-line -- Twitter's manifest destiny, if it's to be successful, is what Biz Stone said the other day in London -- to be the news system of the future. Now take a step back from that, and ask what Twitter has to do, what they have to change, to get there. It might be pretty radical, but it's still not too late, believe it or not. And when we look back in 10 years, with the benefit of hindsight, I'm pretty sure it'll seem to have been the right thing to do.

 11/26/2010; 1:27:52 PM.   .

I was telling a friend visiting from Berkeley that one of the nicest things about New York is restaurant delivery. You can have anything you want almost any time. It's like room service in a hotel, but better, cheaper and faster, with more variety. And with the advent of delivery.com and seamlessweb.com, you don't have to wait on hold to explain your order to some guy who doesn't care and doesn't speak English. It works because there's lots of competition. That doesn't mean, of course, that the food is good -- it might just sound good in the writeup. The other day I saw there was a new Italian place on delivery.com, at triple the price of my regular place. Decided to try it. After all -- maybe there's something really great about a $16 spaghetti and meatballs -- when the price at Pizza Mercato is $6. Turns out not so, the $6 plate is better and bigger than the $16 one.

Thanksgiving is when you give thanks. So thanks for food delivery.

Thanksgiving posts are a tradition here on scripting.com. I might have skipped a year here and there, but ever since the beginning, in 1994 (arrrgh!) the one holiday I usually didn't miss here was this one. It's a good holiday for American bloggers. It's a holiday we share with our friends to the north, but they for some reason think it should be a few months before ours.

Thanksgiving is an American thing, with a universal theme -- thank you! We're not celebrating one religion or political philosophy triumphing over another. There's something phony (to me) about the good cheer people show around Christmas. New Year's Day is good, but it's kind of arbitrary. But giving thanks -- it's a good thing to set aside one day just for that purpose. The end of November seems just the right time. It's a clear time of change and in the change, there is a lot to be thankful for. Sometimes we forget.

Recently I read a story about the inventor Buckminster Fuller, who at age 32, despondent about business failures, contemplated suicide. He decided that instead of dying, he would live the remainder of his life as if he had died. I wonder if this sounds like a foreign concept to you, because it's familiar to me. In 2002, I came home from the hospital to a house that felt like it belonged to a dead relative. I recognized the possessions as mine, at an intellectual level, but they seemed from some other person's life, as I had recognized the things in my grandmother's house after she died. These things are all familiar, but the person they belonged to is gone.

After such a shift your priorities change. Before that, I cared a lot about what people thought of me. I still do, but there's a twist. As I'm processing the insecure feeling that comes from disrespect, I remind myself that I'm gone, I'm dead -- the person they're dissing doesn't exist. Of course they forgot about me, I'm dead. I know it must sound weird, but that's where I'm at. If my Wikipedia page is wrong, well, no one is going to care after I'm gone, so ipso facto, no one cares.

Is ipso facto a real phrase? Have to look it up.

So how does one chart the course of a dead person?

The body has to do something every day. When a dead person wakes up in the morning, what's the first thing on his to-do list?

And the ego isn't really gone, but now it can be reasoned with.

Hard to write about, except on this one day. Because all there is to say is that I'm thankful for this day. I have no future and I don't remember my past. This is all there is. Finally those words make sense, in the way a mathematic theorum makes sense. No matter which direction you approach it from you get to the same place.

Funny, even though the past is a fiction, I still remember a lot of it.

Bob Hope, who lived to be 100 years old, sang about this in Thanks for the Memories.

So if we can be thankful for memories, let's try to keep the sweet ones, and keep the lessons learned from the not-sweet ones.

The great thing about memories is that every year there are more of them.

Moving across the country from a big house to a small apartment, from an ideal climate to one of extremes, time slows down. Thanks for that. It's hard to believe that we celebrated last year's Thanksgiving in Berkeley. This year -- Brooklyn.

Another year more to be thankful for.

I love the story about the guy who enjoyed his TSA patdown so much he wanted to do it again. That's the American can-do spirit. Awesome.

Thanks for the winter that's coming. I love walking in the city when it's cold. Bundled up with only my eyes and mouth exposed, each breath brings cold freshness into my core. My eyes see other bold explorers dressed as I am. Hello there. If there's slush on the ground (coming soon I think) every step is taken with thought. This let's your mind drift in interesting ways while your body is occupied with movement and safety.

Thanks for the bike riding of the summer. I thought I would do it through the winter, but it doesn't seem to be working out that way.

Thanks for living as if you were dead, I think that's not just a good place for a person to be, but it might be a good approach for the country, even the world, to adopt. It looks like the US is approaching the end of our empire. From here-on, we have many equals -- both in economics and in war. Maybe even "equal" is a lot to hope for. Even so, we flirt with disaster, seem to welcome it, judging by the leaders we elect.

But are the woes of one country anything compared to the brink we, as a planet, find ourselves on? It would be better if we gave credence to the idea that we live on a dead planet. If not, as a person who is on the brink would, we should prepare our papers and archives for the obvious and inevitable outcome.

But it's 2010 and we're still here.

Still here but dead. What a world!

 11/25/2010; 3:19:47 AM.   .

I moved into a FIOS-capable apartment in Manhattan a little over a month ago, and have had a chance to get accustomed to having FIOS-level performance from home. It's pretty nice!

I especially like the software in the Verizon router. As someone who has pretty much mastered port-forwarding and dynamic DNS, I appreciate the completeness of the UI. It can be hard to find some functions, but for most of them, it's all where you think it should be.

But then I wondered, as I started to set up a small server to run out of the house, taking over some of the work of my EC2 servers, why don't they give everyone a few gigabytes of static HTTP storage, right in the router. It would interface as a local-net-accessible folder. Any file you copy into the folder would be available over the net. Easy UI. A bit of the cloud right there on yer desk.

 11/25/2010; 2:56:17 AM.   .

I'm once again looking into using Posterous for podcasts. I see from the FAQ that they do them, but the question is, can it fit into our process for RBTN. Which goes like this...

1. Jay creates a post but it's not published. The post contains the items for this week's show. We go back and forth, then we do the show, and after it's over, he does a light edit to reflect what we actually talked about.

2. I prepare the MP3 that Adrian, our engineer, gives me. That means verifying that you can hear us talking, adding metadata, and uploading it to mp3.morningcoffeenotes.com. It's really important that all the MP3s be served out of the same folder. I've learned the hard way that if you don't do that, a few years later half your shows are somewhere else that has gone away or you don't remember where you put them. So I keep disciplined. This means that uploading the MP3 to Posterous is not an option.

3. Then I add the MP3 link to Jay's blog post so the CMS can scrape it. It's a totally wrong UI, but I didn't have any say in it, so don't blame me, I'd much rather just enter the URL in a dialog and have the CMS render it as an icon that communicates clearly to the user what's going on. Having an in-browser player is a pretty good idea too, and would just make our UI match that of the professional sites. All of these guys make us look bad, but meanwhile we've been upgrading the production of the show, and I think we've gotten a lot better at what we do. While this is happening our technical team (wordpress.com) has been making us look worse and worse. I know Matt's a business guy, this is a business issue. We care about our users, but we can't do it if the tools vendors don't care about theirs (i.e. us).

Anyway, I haven't really figured out how to or if I can make this work with Posterous.

Wordpress, Tumblr, Posterous.. are there other choices? Is anyone out there in PodcastLand happy with their CMS? Please let me know.

 11/23/2010; 12:32:00 PM.   .

As you may know, I do a weekly podcast with Jay Rosen called Rebooting The News.

This is a heads-up to people who subscribe to the podcast. There is a problem with the feed that will cause some subscribers to not get every episode. (It could be a majority of subscribers not getting any episodes, depending on how their clients are programmed.)

Technical: The problem appears to be caching. After a post is published, it builds the page, and caches the HTML. Somehow this is done before it realizes there is an MP3 attached. Eventually the cached feed expires, it rebuilds the feed, and now the MP3 is linked in as an enclosure. How long before this happens and how many clients have read the post and stored the guid and assume there is no MP3? Unknown. But a correctly written client that scans frequently will miss the MP3. My client does, for example

Until this problem is fixed, you should manually visit the site periodically to see if there's a new MP3 available and download it to to your listening device.

This is a repeat of a problem that hit our podcast in the beginning of the year. It was fixed, the feed was working properly for a few months, but it's broken again. The Automattic people are aware of the problem, and hopefully will have a fix soon. As I understand it, all podcast feeds on wordpress.com are broken, btw -- so if you're hosting a podcast there, you should be aware of the problem.

We continue to use WordPress because Jay is very comfortable with it. We checked out Tumblr, and it's not any better at podcast support than WordPress is.

A personal note, none of these programs work as well as the tools we had when podcasting was booting up in 2001-2004. Had our tools been this bad, there would be no podcasting, I'm sure of it. As these services have scaled, important features are breaking. This of course totally sucks.

 11/23/2010; 9:03:38 AM.   .

When I publish a picture from my iPhone, I:

1. Take the picture (that's the fun part).

2. Click the send icon at the bottom of the picture.

3. Choose Email from the dialog.

4. Type FU to get the first two letters of my flickrupload email address.

5. Wait because the iPhone is ridiculously slow at recognizing this sequence of clicks.

6. Click in the subject area of the envelope.

7. Type the title (a necessary step, no matter what method is used).

8. Click Send.

9. Choose the resolution from the dialog that pops up.

And that's just the beginning of the trip my picture is going to take. Now it has to get from Flickr to Twitter somehow. For that, I usually wait until I can get to a laptop with a net connection.

It'll be interesting to see how this compares to the way pictures are shared from the road in the future. Hopefully most of the steps will be gone.

 11/22/2010; 7:33:57 PM.   .

From the top of the Empire State Building.

 11/20/2010; 10:11:55 PM.   .

I've had a persistent problem with static hosting, going back to the beginning of scripting.com in 1995 or so. That's a long time to not solve a problem. While all the incubators start new tech companies with viral business models, one of the most basic ideas about the web, a folder that maps onto a website, still requires a degree in bicycle repair (not quite rocket science) to keep going.

I talk to my friend Chuck Shotton about this all the time. Chuck used to write web servers.

I say to Chuck one day we should knuckle down for a month and do an completely brain-dead simple self-hosting server, running on a Mac probably, cause it comes with Apache built in and turned on. We never seem to get around to it.

Meanwhile, Amazon made it so that if you stand on one foot while touching your nose with the pinky of your right hand and say Wild Beatnik Pie! three times you can host a static site in S3, which seems to have been made to serve static sites. My friend Joe Moreno has mastered this art, and posted a howto about it. I am going to proceed to follow his directions and set up a simple site in a S3 bucket, one that very rarely changes, to see what happens.

Joe Moreno: How to host a static site in S3.

1/2 hour later, I think I've set everything up right, but I get a "Sorry invalid request" when I try to visit www.smallpicture.com, a site I set up following Joe's instructions. Maybe it takes a while for things to show up?

When I tried to go to dsxuvulkqwac7.cloudfront.net it correctly linked to the index file in web.smallpicture.com, but I got an "access denied" from (what appears to be) S3. After changing the permissions the index file correctly displays, but I still get the invalid request message when I go to www.smallpicture.com.

I think the problem is in Joe's step 2. I changed it so that www.smallpicture.com is a CNAME for s3.amazonaws.com. I bet that fixes it and www.smallpicture.com will now correctly show the small Mona Lisa.

 11/19/2010; 6:35:20 PM.   .

Saw this headline roll by and had to remember to breathe. Until that moment I thought of Tumblr as a tiny company operating upstairs in a random midtown office. David Karp, its founder, is brilliant, and the site is growing at an incredible rate, and now the company is valued at $135 million. Oh to be young, talented and rich in NY in 2010.

Instant comments say its more bubble bidding, but it might not be. I hope Tumblr opts for a WordPress-style business model known as "freemium," where they give away the basic site, and charge for the extras. That will keep their interests nicely aligned with their users'. I think long-term, companies like Twitter and Facebook will have big problems as they have to narrow the margins for their participants, always competing with more members of their community, forcing them to either disappear or look for greener pastures. I don't see any clear line of places they can't go. But if Tumblr charges for sites, then its users are customers, and that's something that we're all comfortable with because we're either customers or have customers in the rest of our commercial lives. The user-as-hamster model is growing more uncomfortable all the time.

Anyway, it might not be obvious at first that this is not great news for the publishing business, as it's currently configured. Flush with cash, companies like Tumblr, Foursquare and others are mining tech talent in NYC, and bidding up the already-high prices for engineers. If you're a newspaper or magazine publisher in NYC, and you've been employing technical people, watch out -- you're competing with employers with newly deep pockets. And don't forget Google is (rumored to be) buying a huge office building in Chelsea, near the meatpacking district. It's going to be filled with technical people too.

Of all the tech wunderkinds out there, it's mainly WordPress that sees the media as customers. It probably would be a good idea if they had some competition.

 11/19/2010; 1:56:38 PM.   .

I've been hooked into the Hacker News firehose feed now for about a week and I love it.

I haven't seen any spam or abuse. It's a community that seems to value news and ideas. A lot of is opinionated, and some of the opinions are disagreeable to me, but the whole thing is very good. And I learn a lot from people I disagree with, anyway -- so it's all good.

I push a handful of the links I like from HN to Twitter, so you all are getting the benefit of my reading this new source.

Highly recommended!

 11/18/2010; 4:28:37 PM.   .

I saw this cartoon comparing SOAP to REST in the Hacker News firehose feed, and wondered why Fox-News-style politics has to be part of technology discussions. Where is the religion coming from? Don't people want to use the best tool for the job? Would people willingly take a big step backwards? The answer to the last question is absolutely yes. It happens so often.

Okay first -- let me get this out of the way -- SOAP is a mess. That's a strong statement coming from one of the guys credited with designing it. But I left the design process early because I didn't like the mess the big companies were making. I was quite vocal about it.

But the first versions of SOAP were really simple and easy to implement. You could do a full implementation in a weekend. The later versions were so complicated that you could never finish an implementation. No matter how complete you thought it was, someone could throw a call at your server that it wouldn't understand. A cynical observer might think the big companies liked this aspect of SOAP because they could claim to be standard-compliant without all that inconvenient interop!

Luckily the simple version of SOAP was published, in 1998, and a vibrant community developed around it. There are implementations in all the major development environments. It's baked into Python and the Mac OS. And a set of APIs for blogging tools was developed and were very widely deployed. All the major blogging tools supported the protocol, including WordPress which is so popular these days. I host the XML-RPC site, and to this day it gets a lot of traffic. A lot of people are, very quietly, using it.

So when people compare SOAP to REST, and not to XML-RPC I wonder if they know that REST is missing something that it should always have had, a standardized way of serializing structs, lists and scalars. That's something XML-RPC (and SOAP) have always had, and any discussion of these technologies should include that advantage relative to REST, which forces you to cook your own serialization with every API.

Because other developers like them, I implement REST interfaces all the time, and have been doing it for years. Every time I have to re-implement a layer that's provided built-into XML-RPC and SOAP. True, there is no single SOAP, and I would never, unless forced to, choose to implement an API in SOAP. But XML-RPC is a whole other matter. It's simple, easy, and it interops.

The need for a common serialization was so great that people are dumping XML in favor of JSON to get it. And when they do it, they talk about the incomprehensible complexity of SOAP. Too bad, because XML-RPC never had that complexity.

So come on engineers, stop being so Fox-like and open your eyes. Maybe we can do better than we have been doing.

 11/18/2010; 4:02:53 PM.   .

I'm a big believer in designers, programmers, writers, artists, news people all working together.

So it's in the spirit of a programmer looking for help from designers that I ask the following question.

What's the best way to display a River of News in a browser?

A river is a strict reverse-chronology of news items that conform more or less to the elements of a RSS 2.0 <item>. So you could have a title, link, description, publication date, link to comments, categories, or enclosures. There are a few other elements that can be in an item. For a full list, check out the RSS 2.0 spec. It's written so that a designer can make sense of it.

Here are some examples of rivers:

1. nytimesriver.com -- designed originally for Blackberries.

2. Arc90's rendering of the 2006-era river thanks to archive.org.

3. A screen shot of the Radio 8.0 river, in early 2002.

4. east-village.org, produced by River2.

5. Twitter is a river.

7. Les Orchard's news page (the descriptions would have to be initially expanded to make it fully a river).

8. Glowdart is very interesting. I'd like to experiment with a UI like this, but one where the user could give it any number of feeds to follow. In other words it's a nice start for a generalized RoN aggregator.

The design challenge is this. GIven the latest HTML techniques, do a mockup of a great River of News. If it's really something new, I'll put the software behind it and make it live.

 11/16/2010; 7:13:36 PM.   .

A very simple observation.

When the Republicans are in power and running up huge deficits, the issue of deficits is nowhere in sight.

During the last Bush presidency we should have been talking about deficits and the national debt and what we were passing on to future generations.

We were fighting two wars and had tax cuts at the same time. A very unusual confluence, and a really powerful way to accelerate debt. Usually wars are times of sacrifice, with rationing and high taxes, compulsory service. Even so we emerge from wars usually (if we win) with huge debts to pay off.

Now when the Democrats are in power, and the issue is (finally!) investing in America, all of a sudden the deficit is the big issue.

And who's making it such a big issue? The same deficit-loving Republicans (when they're in power).

Reality: No matter who's in power, what we as a country spend money on should always be an issue. Whether we go into debt or not is moot. There will be times when we will go into debt. It's as natural as the ebb and flow of the tides.

If the decision is spending our grandchildren's money on blowing up Iraq, or making them healthier or better educated -- I don't think it's much of a choice -- do you? Yet the Republicans seem to spend freely to kill Iraqis and fight against investing in America. We build new infrastructure in Iraq, and then blow it up, but we won't build new infrastructure in the US, at all. This is lunacy. We are collectively out of our minds if we let this happen.

Meanwhile our competitors invest in the health and education of their grandchildren. The next generation of Americans won't thank us for not investing in their future, they'll be the menial labor of their generation -- they will curse us.

"Running a deficit" is just another way of saying "investing." The hype is that we have to balance our budgets at home, so the government should too. But that's not true -- most of us run deficits, esp when we're young. If you have a mortgage, you're "running a deficit." And it's a good idea. When you're young you haven't had the time to accumulate enough money to buy a house, so the financial system lets you buy it a little at a time. Really, there's nothing imprudent there. If you're young and you're against debt, and don't have any -- you probably had rich parents.

Student loans are also a good idea if it means you'll earn $120K instead of $30K per year. You can pay off the loan with part of the higher salary. If you don't invest, the only way to grow is by windfall -- and our windfall in America has run out. We don't have great natural resources, we're a resource-importer. That means unless we want to spiral downward, we must invest. And that's why we have deficits.

We grew up thinking it was our birthright to run everything and own everything. The truth is, that was a bubble that had a lot to do with geography, and our advantage is largely over. That's not any party's fault, much larger forces are at work. Now we have to think to keep going, and think very well. And at just this time, thinking is something it seems people don't want to do.

PS: As with deficits, there's nothing wrong with earmarks. If the government decides to spend money, either the executive branch or the legislative branch decides how to spend it. (Occasionally the judicial branch makes spending decisions too.) If it's the legislative branch deciding, that's an earmark. If it's not an earmark that just means an employee of one of the departments, likelly not even an elected official, is making the decision. To campaign against earmarks, as the Republicans are preparing to do, is to assume the people have no idea how government works, and that no one (i.e. the press) is going to clue them in.

PPS: Love this story about the newly-elected Republican from Maryland, who campaigned to repeal health care reform, pissed off that his health care (free with his new job) doesn't start for a month after he's sworn in. It's okay for the rest of us to pay for his health care, but if we want help with the insurance companies, that's too expensive! These Republicans are such greedy hypocritical and heartless assholes.

 11/16/2010; 10:01:41 AM.   .

Just happened to stop in at apple.com this morning and saw this teaser.

Now what could it be?

I rolled it around for a bit, and decided to put a stake in the ground. I'm probably way wrong, but maybe there will some other interesting ideas.

I think we're going to see software for the iPhone that turns it into a remote control for iTunes. The two devices would connect over the LAN. And it will do things that a remote has never been able to do, because it has a software-controlled display and it knows what content is on your server that can be played. And it knows about your playlists.

Apple has always shipped fairly useless dinky remotes with Mac Mini's and Apple TVs (I don't think iMacs or laptops come with them) but I've never been able to use them. And likewise, the full-screen software for controlling iTunes is (I find) cumbersome and difficult to control. Seems like the UI for remote control belongs on an iPhone. And btw, there isn't an Android version of the software. Sorry.

That's my wild speculation for today!

PS: I like the idea that Apple is going directly to the customers instead of going through the press. No reason for a middleman here. They are perfectly capable of teasing on their own.

PPS: Names for such a product: iMote, airMote.

PPPS: As has been pointed out in the comments, they already have such a product. Never mind.

 11/15/2010; 3:01:52 PM.   .

I went to dinner last night with a couple of old friends who live on the Upper West Side. As I left my apartment, I thought I might text them to say I was on my way. As I walked to the train station I thought about an app I wished existed.

I'd like to connect my iPhone with the computer at my host's home, just for the next 1/2 hour, so they could watch my progress. That way if I was going to be ten minutes late they would know as soon as I knew. Obviously this is an opt-in thing on both ends.

Similarly if I'm going to visit someone who's going to pick me up at a train station. I used to ask people who were visiting me in Berkeley to text me when they got to the MacArthur station on BART. That was approximately when I would have to leave to meet them at the North Berkeley station. This could be automated.

Is it possible this app already exists?

 11/15/2010; 10:57:10 AM.   .

I woke up this morning to stories in my aggregator about path.com. I had heard about it before their launch, and was intrigued by the name. Turns out it's yet another instant photo sharing iPhone app. I have limited enthusiasm for them, I already use my iPhone, extensively, with Flickr and Twitter.

Even so, first thing this morning I signed up for a Path account on my iPhone.

After entering my name and email address, gender and password, it asked if it can use my location. I said yes. Then I went to the People section to start looking for friends to share my pictures with. I was astonished to see a list of suggestions, all of whom are people I know. I was confused. How could they know I know all these people? I jumped to an incorrect conclusion, they were all following me. I smiled -- it's really cool that all these people, some of whom I haven't spoken with in years, are following me on Path. After happily adding eight people (noting that Andrew Baron had signed up twice, with two different email addresses), I realized that can't be it. Some of these people are so totally offline they could never be using this app on its first day of public existence.

So I went to Twitter and asked if anyone knew how they were doing it.

fr8d had the obvious (in hindsight) answer: They looked in my iPhone's address book.

I never said they could. What else did they do with my contacts? Send a copy to their server for safe-keeping? Foolish me, but I thought that was my iPhone and my contact list. I paid huge money for the iPhone, so it's not like it could be anyone's "business model" to use that data. But now, as far as I know, some unknown startup in California has all my data.

As I fumed, I said -- The tech industry is a virus.

An analogy. I'm standing on a subway platform and someone behind me is reading my credit card numbers aloud. I turn around and see they have my wallet. At that point does it matter if they're going to use the info to buy some goodies at Crate & Barrel or is the damage already done?

It's like spammers took over technology, like the pet food guys did in 1999. Everyone has a scam. This year the scam is to grab all the user's data and resell it. It's gotten to the point where it's a risky proposition to try out a new iPhone product.

Another example. When I realized that any random Twitter app who you give your credentials to can download all your private direct messages, that was the end of me using Twitter apps that want credentials. Meanwhile the team at Twitter Corp has always had access to this info. Who's to say their interpretation of one of their terms of service is that they get to analyze and mine every bit of text I enter into the system even text that's only meant for one other person to read?

It's bad. That's the point of this message. I'm trying to end it on a positive note but I can't think of one.

 11/15/2010; 9:24:28 AM.   .

Great piece on Epicenter explaining, plausibly, how Facebook could grab Google's ad business by using data they gather better than Google does. Further, even though Google does accumulate some of this data, they can't use because they've promised not to.

I doubt a few years ago if Google thought this kind of route-around was possible.

The same thing happens on the journalism side of things as well. A few years ago CNET probably felt they could not be routed-around, but they were. It happens regularly, and will certainly happen again.

I have a theory that you can avoid being routed around if you don't try to erect the barriers at all. If you welcome all comers, and try to learn from them, including your competitors. In such a world, Google would have tried to do what Facebook and Twitter do as soon as they came around, without trying to use their installed base advantage to compete. I don't think that organizationally they could use their own people to do it, rather they would invest in the newcomers, aggressively.

One tech company that appears to get this is also a generally quiet company that doesn't bluster much, and tries like hell not to take sides in the technology wars. For 10 points, which company is that? Click on the plus next to this paragraph in the blog post for the answer.

 11/14/2010; 4:41:06 PM.   .

Fred Wilson has a post today about the explosion of platforms that developers may choose to support.

It's a race to see which one gets the killer app, the one everyone has to have, and then the other platforms will drop off, and there will be a consensus platform, the one "everyone" develops for. I hope, for the sake of all of us, that it's a platform without a platform vendor -- that no one gets to decide who has access to the market and who doesn't. Of course if that ever comes about, and the vendor throws around their weight too much, that's when the DoJ steps in, assuming the superrich haven't completely bought the government by then.

Anyway, it seems to usually happen that way. There were lots of personal computers until Visicalc came along, then the consensus was the Apple II. Until the IBM PC came along and everyone wanted to run Lotus. Then Windows and Excel, and the web and Yahoo, Amazon, Google, etc.

 11/14/2010; 3:34:55 PM.   .

Andrew Keen tweets: "While Facebook wants all of us to publicize our personal data, its strategy is to privatize its data from Google."

To which I responded: "Welcome to the tech industry! Welcome to your hamster cage!"

This isn't the first time I've written about hamster cages, but I haven't explained what I meant.

They make a wide variety of colorful and fun cages for hamsters that are designed to keep the hamster, and their human owners, entertained for hours. When you get tired of one, you can buy another. It's looks great until you realize one day, that you can't get out! That's the whole point of a cage.

Remember how they used to say: "If it sounds too good to be true then it probably is?" They still say it.

Another one: "There's no such thing as a free lunch." Exactly.

When they say you get to use their social network for free, look for the hidden price. It's there. They're listening and watching. It's pretty and colorful and endlessly fun for you and your human owner.

 11/14/2010; 3:22:56 AM.   .

The feed is more or less done.

http://static.scripting.com/hackernews/rss.xml

Each item points to the article being linked to from the Hacker News entry. The link points to the HN discussion page.

The feed is realtime, it has a functional element, and the notification server works (at least for now, knock wood, praise Murphy).

I am following it in my personal river. It's a lot of news, but I like having it there.

You're welcome to poll it, but it only updates at most once a minute, so there's no point in polling more frequently.

The source of the flow is http://api.ihackernews.com/new.

PS: Lots of interesting comments about the feed over on Hacker News.

PPS: The Twitterfeed folk are using this feed to link the firehose to Twitter.

PPPS: Even better, they say they're going to hook into rssCloud!!

 11/13/2010; 11:16:32 AM.   .

Over 40 percent of all New Yorkers today were born outside the United States. This was true at the beginning of the 20th Century, but there was very little immigration after that. Only recently has the city re-opened for immigrants.

Immigrants are "colonizing" neighborhoods that, in the 60s, 70s, 80s and 90s, were in decline, thanks to urban renewal and redlining.

I can testify to that. The neighborhood I grew up in, Flushing, has become almost entirely Asian -- Chinese and Korean mostly. Much bigger than the Chinatown in Manhattan with lots of great places to eat, and what appears to be a booming economy. Russians have settled in Brooklyn, Indians in Jackson Heights.

I left New York in the 70s, and I never wanted to come back. I remembered big parts of the Bronx reduced to rubble, and the rest of it on its way. When I left, Brooklyn was in very bad shape. I always assumed my rejection of NY was more about me than the city, but it turns out that's not true. The city was in decline, it's only recently coming back. Things work now that never did when I was young. It's mostly safe, people go places that would never have been considered safe when I was growing up. Haven't heard anything about strikes. In the 60s and 70s all kinds of city workers, sanitation, transit, would strike.

You can definitely live in NY without a car. I've only driven my car twice since April, and that was just to move it from one place to another. I don't think there are any other places in the US where you can get by without a car.

However, I wish they could reduce the presence of cars in the city. I'm pretty scared to ride my bike here since the accident in September. That was caused by a car. I now live in the middle of the island, far away from the Hudson River bikeway. I know it sounds radical, but I wish we could give one avenue to bikes, and completely ban cars, buses and trucks. It would take a revolution to make that happen. But it would be transformative. Not proposing it, but thinking "what if."

Business-wise, there is a huge opportunity here, but the more I get to know people in business here, the more I hear them clinging to the ways of the past. It happens in California too. But the sooner the media, or some portion of it, fully embraces the new channels of communication and cuts its dependency on the distribution system of the past, the sooner New York can resume its central importance in the US economy. It could happen. It could be an extension of the rebound we're seeing here now.

 11/12/2010; 1:29:46 PM.   .

Got an email from a developer friend asking if RSS had rebooted yet.

Here's how I responded..

Well it never actually needed a reboot -- with billions of feeds updating all the time, it's pretty much like Facebook, without all the money.

 11/12/2010; 12:50:07 PM.   .

Dave Winer, 55, is a visiting scholar at NYU's Arthur L. Carter Journalism Institute. He pioneered the development of weblogs, syndication (RSS), podcasting, outlining, and web content management software; former contributing editor at Wired Magazine, research fellow at Harvard Law School, entrepreneur, and investor in web media companies. A native New Yorker, he received a Master's in Computer Science from the University of Wisconsin, a Bachelor's in Mathematics from Tulane University and currently lives in New York City.

"The protoblogger." - NY Times.

"The father of modern-day content distribution." - PC World.

One of BusinessWeek's 25 Most Influential People on the Web.

"Helped popularize blogging, podcasting and RSS." - Time.

"The father of blogging and RSS." - BBC.

"RSS was born in 1997 out of the confluence of Dave Winer's 'Really Simple Syndication' technology, used to push out blog updates, and Netscape's 'Rich Site Summary', which allowed users to create custom Netscape home pages with regularly updated data flows." - Tim O'Reilly.

scriptingnews1mail at gmail dot com.

Facebook

Twitter

Friendfeed

Changes

Droidie

East Village

Frontier News

Great VaVaVoom

Leon Winer

Links

Listings

NY Times River

OPML Editor

opml.org

Outliners.com

Protoblogger

ReallySimpleSyndication

Realtime RSS

Rebooting the News

River2

RSS spec

rssCloud

Tech River

Unberkeley

XML-RPC

My 40 most-recent Twitter links, ranked by number of clicks.

People are always asking about my bike.

Here's a picture.

FYI: You're soaking in it. :-)