Archived entries for Current Events

Twitter Faster than Reality

LA shook at 11:42:15 today according to the official record from the U.S. Geological Survey. But according to [a report of] Twitter activity today (by the tweetip site) it happened 43 seconds earlier at 11:41:32 (adjusted for time zone).

tweetip

(graphic snagged from tweetip site)

That Twitter routinely breaks news fastest is often discussed, notably in the wake of the May quake in China.

Today the AP’s wire posted news of the earthquake 9 minutes after it happened. 9 minutes is fast. Negative :43 is amazing.

(Yeah, yeah. I know. It’s explainable as an accounting error in twitter’s api or tweetip’s processing. But the point remains that twitter is always on the scene.)

Twitter and Summize. No worries.

There are rumors that Summize has been acquired by Twitter. It has people chattering.

Some worry that the acquisition will hurt the effort to make Twitter scale. It can’t and won’t.

I believe Twitter’s engineering team is headed up a mountain (they need to switch architectures at a low level), but that they finally know which mountain. True, it’s a tall mountain not quickly climbed. But they finally know their problems and have people in place. Better days ahead.

Others worry that Twitter’s scaling ills will infect Summize. I don’t think that’s possible because they are distinctly different engineering problems. Summize is “fresh search,” an understood and known problem that Summize apparently designed for from the beginning. Twitter, in contrast, evolved a product into a service that no longer matches their architectural model. It didn’t start out as (and therefore wasn’t built to be) a massive-to-massive (when each massively is unique, personal, exponentially expanding) real-time messaging protocol. I believe architectures exist for that problem space, but unfortunately that’s not how Twitter was initially built.

Put briefly, Twitter’s already on the path to health and Summize is immune from Twitter’s disease, so it should all work out fine.

While they are different systems, they may be complimentary. Jettisoning Twitter’s track and reply functionality to Summize’s infrastructure may offer Twitter engineers the headroom they need to roll updates into Twitter’s codebase with a bit of a cushion.

The Big Picture: The Fires

Eric Miraglia, my friend and YUI teammate, tipped me off to a great blog last week during the show-and-tell portion of our weekly staff meeting. It’s a photo-journalism blog called The Big Picture. It’s published by Boston.com / The Boston Globe.

As the name implies, they publish big photos. Not thumbnails or small one-column photos like most news sites (and sites in general), but true large format photos. Generally 990×660. It’s remarkable the greater impact that larger photos can have.

Today’s feature is on California’s Continuing Fires.

There are a LOT of fires burning. Coming home from Golden Gate Park yesterday after Tasty, we crested Twin Peaks and had an eastern view of the entire bay as we drove on Portola Drive. In near unison we all noted the “fire smog.” The air is thick with smoke, even in SF which is currently fairly removed from the fires.

About a month ago, my buddy Matt’s house in the Santa Cruz mountains came within a kilometer or two of being engulfed. If the winds had been normal his house would have been gone. But luckily the winds were anomalously blowing the opposite directly. They evacuated, but were spared.

About 10 days ago, my friend Jud’s mom was evacuated from her home in the Brisbane hills just a few 2 or 3 miles south of SF. She avoided disaster, too.

Last week I flew out of SFO. All flights were delayed because of lack of visibility due to fire smoke in the air across the whole region.

So, take a look at the fires through the Big Picture lens to get a better sense of what’s really going on, and the amazingly tough and dedicated firefighters. There are more than 20,000 hard-core people out there fighting to get it all under control.

Here’s to them.

And here’s the feed for the Big Picture so you can add it to your reader.

Liveblogging Google App Engine release at Campfire One at Google

Liveblogging on Twitter at http://twitter.com/natekoechley

everything in this article is my paraphrasing of speakers’ presentations. not my own words.

(Video coming soon.)

  1. We run web applications. We’re only focused on this narrow goal.
  2. We handle the entire lifecycle of an app.
  3. Apps are run on Google infrastructure.

“It’s hard, but it’s worth it for us.”

“For the first time you can use the same infra we use…Auth, GOS, BigTable”

The Stack

  1. Scalable serving infra
  2. python runtime
  3. SDK
  4. Web based admin console
  5. DataStore

Demo: App from scratch in 8 minutes.

More details

  1. Scalable Serving Infrastructure: fault tolerant (redundant). Fluid: don’t need to schedule needs up front… more servers come online dynamically.
  2. Python Runtime and Libraries. All tools are generic, so new languages can be dropped in later. Python used in same python available otherwise. Goal: you can use any language eventually. We don’t want to limit you.
  3. SDK: Environment to develop apps locally. Avail for Linux, Mac, Windows today. (But can probably work anywhere.)
  4. Admin Console: web-based admin console. (Looks like google finance meets google analytics.) Tools for request logs. Data explorer. Usage/quote numbers. App-version balancing. Can hook up domain (don’t need to run at *.appspot.com).
  5. Scalable Datastore. Schemaless object store. Not a clustered sql thing. Instead based on BigTable. (Whitepapers online.) Horizontally scalable. Reacts to hotspots. BigTable instead of SQL is a big change, and may take some time to get used to. But we think you’ll come to like it. Schemaless means you can add a new datatype or entity whenever – no need to update your schema.

Now we’re looking at a Datastore Model Class.

GQL Query example

SELECT *
FROM Story
WHERE title = 'App Engine Launch'
AND author = :current_user
AND rating >= 10
ORDER BY rating, created DESC

Other Notes

Mail Sending API

no setup needed.

Make HTTP Requests

Authenticate with Google Accounts

Frameworks

The whole Django framework.

Guido van Rossum: Creator of Python and member of Google App Engine team

My passion is making life easier for developers. With python i’ve done that for decades. Now i’ve joined GAE team. Excited by potential. (and that python was first picked)

First time that GOogle has let third-party people run software on their infra. That’s fundamentally a big deal.

8:13 PM “We’re offing 100% of the python lang.”

8:14 PM – we don’t offer threads, but you won’t been it because of our scalable arch.

GAE uses a quota system so nobody monopolizes the infra.

me: if it’s so scalable, why do they need the quotes?

What’s Next?

  • large upload/download support
  • purchase additional capacity
  • other language support
  • offline processing.

Live on Yahoo! Live

Carbon Neutral Purple

I know there’s a bit of a backlash against Green because its so trendy lately, but I can easily put that aside and be happy that things are changing. That takes on special meaning today because I just saw that Yahoo! is quickly following promises with real action, and making what seem to be excellent, well-researched green choices.

When Yahoo! committed to going carbon neutral in April, we knew it would be a global initiative. … After much due diligence, Yahoo! has decided to offset its 250 thousand metric ton carbon footprint from 2006 through hydropower in rural Brazil and wind turbines in India. We’ve partnered with EcoSecurities and CantorCO2e, who helped us source, vet, and execute these projects.

(Some are still skeptical about carbon offsets, but I see any step as a great early step.)



San Francisco, California | Creative Commons By-2.5 License | Contact

RSS Feed. This blog is proudly powered by Wordpress and uses Modern Clix, a theme by Rodrigo Galindez.