Tuesday, January 24, 2012

MongoDB Tip #2 - updating fields via Java/Groovy (Bicycle Shop Example)


First post of the new year, 2nd MongoDB tip... here we go!

Let's say that you're writing software for a bicycle repair shop. You need to update the "flat tire" status for a handful of customers because those tires were repaired by the repair shop team.

Using native MongoDB Javascript, you might craft the following:

db.bicycleJobs.update({jobId : { $in: [ 110234, 110433, 110511, 110766] }}, 
 {$set : {flatTire:false}}, false, true); 

Yes, we could cruft up a javascript file to do this, but you want to grab the customer job numbers (e.g., 110234 above) from a file or your inventory system.
So you decide it would be much more convenient to do it in Java or Groovy. Interrogating the native mongo statement we used above, you can probably guess what the update method signature looks like according to the Java Driver for MongoDB.

customers.update(document1, document2, flag1, flag2)
document1
This is our "select" that identifies the target rows based on job numbers in the provided list.
document2
This is our "action" document that actually states the operation to take place. Here we "set" the field called "flatTire" to false.
flag1
This is the "multi" flag. It indicates that the set operation should be applied to all rows that match the results of document1, not just the first match, which would be the default behavior. Having been burned in the past, that's what I call a nice default behavior!
flag2
This is an "upsert" indicator, meaning if there's no match to be had according to document1, then create the row, initialized with these two fields: jobId and flatTire. Might look good on the books, but let's keep this behavior turned off since it probably reveals a data entry error (with respect to the correct jobId).

So a more useful description of this Groovy method signature might something like:

bicycleJobs.update(findDoc, operationDoc, multiFlag, upsertFlag)

Now here's some code to tie things together...


def jobList = [] as BasicDBList

jobList << 110234 // until we're reading from a file or other data source...
jobList << 110433
jobList << 110511
jobList << 110766

def findDoc = new BasicDBObject().append('jobId', 
 new BasicDBObject().append('$in', jobList ));

if (verbose) {  // This bit gives us a little debug to see "the before"...
 def cur = leads.find(findDoc)
 while (cur.hasNext()) {
  def customerRow = cur.next()
  println customerRow['jobId'] + ': flatTire: ' + 
   customerRow['flatTire']  + ': customer: ' + 
   customerRow['name']
 }
}

def operationDoc = new BasicDBObject().append('$set', 
 new BasicDBObject().append('flatTire', false));

def multiFlag = true // update all matches
def upsertFlag = false

db.bicycleJobs.update(findDoc, operationsDoc, multiFlag, upsertFlag)

//You could repeat the "verbose" block above to confirm the expected results.

That's it for now. As I build my knowledge with native mongoDB, I'm trying to make sure I know how to do the equivalent in Java/Groovy. Once you understand how to use Hashes and BasicDBOObject operations to build a document, then it appears to be pretty straightforward.






Saturday, December 31, 2011

A definition for "cloud" computing

Before "the cloud," you pretty much new where you data was getting stored. If you were on a business network, you probably knew that your files were stored on your local hard drive or on the network server located in the closet in the office. Or maybe you had heard that "the server" was hosted by a downtown Internet Service Provider company. You may have even heard that it was still your company's hardware. They just kept it cool and in a "cage."

So, with the cloud, how is that different?

If your files are now hosted in the cloud, then you no longer can picture where they reside. That's because instead of having one or two dedicated pieces of file-hosting hardware, your network folks have instead subscribed to a cloud service like Amazon.com. That means your network file serving has been "thrown over the wall" to a stranger like Amazon.com. What machines they use, what technology they use is their concern. All you know is that you still see your files. And you've heard your network folks love it because they let Amazon.com deal with the hardware, the backup responsibilites, the "failover" prevention, the security and so forth. In fact, they love it even more because they know they can keep adding more room for storing more files as the company grows. And all without having to order new hardware.

But what's this "iCloud" thing from Apple?

Well, it's just more of the same, except there's no network group at the office involved. Unlike Amazon.com, you don't have to deal with bringing up "an instance of a server." Instead, it's more of a consumer-focused file hosting service. It's just between you, your Apple device and the iCloud service. Again, you have no idea where they store your images or tunes. They just make sure you have constant access to what you "throw over the wall to them."

So what are the implications?

As long as Apple stays in business, you have "control" of your files. As long as Apple keeps their servers healthy, you have access to your files. You now don't have quite the concern for how big your disk drive is anymore, right? That's because your store your files on the cloud now.

Other implications? Of course, but I'm not in the mood to reflect on the big brother nature of all of this, or the house of cards scenario. It seems that this is direction the forces of business-driver evolution are taking the Internet. It's not an evil plot. It seems to make sense.

David

Thursday, October 06, 2011

An adoptee's perspective on Steve Jobs...

Steve and I went to the same high school, Homestead, on the border of Sunnyvale and Cupertino. He was a freshman when I was a sophomore. Wozniak was a senior. Steve dated my next door neighbor Marla and showed up in a satin white tux with top hat at the 5 year high school reunion (according to my sister Carolyn who was in the same graduating class of '72).

As I tell people, while I hung out with girls, he was more of a "shop" guy. Our paths never crossed. Or they did but I didn't know him from Adam. But later I met him twice, once when he entertained a group of us from SCO (the Unix on Intel company) at NeXT with the purpose of convincing SCO to ship "NeXTStep", a development environment that he was trying to make more pervasive in the Unix world.

It was there, at the NeXT offices near Redwood City off of 101, visiting Jobs with two SCO VPs where I learned what a crutch powerpoint presentations are as well as Jobs' commanding knowledge of the industry. After Steve and I shortly reminisced about the good old days at Homestead High, one of our VPs got up to give him a presentation of SCO and our 2-tier distribution model, of which we were very proud. About to display the 3rd slide, Steve waved him off, asked him to sit down and proceeded to summarize the SCO business model as well as I'd ever heard from our own people. I think it's fair to say that our guy was pretty devastated to be brought down to earth so quickly... especially given how much he probably had rehearsed the days before in order to impress Jobs about SCO's unique Unix-on-Intel business model.

At the end of the meeting, it was already clear to us that he was a little nuts and, in a way, totally out of his league. He was trying to build a consortium without forming a consortium. And consortiums are difficult to control. Steve had never given the world the impression that he was a "standards kind of guy" or capable of adapting to the committee-driven nature of Unix standards. At the end of the discussion when we were trying to understand his real motivations, he finally said (to paraphrase), "Well somebody's got to stop Microsoft!" I kind of shook my head and thought, "this poor guy... he's still driven by his personal quarrel with Gates."

I later interviewed at NeXT to run their product engineering group. I again had a casual conversation with Jobs, but my interview was with Avie Tevanian, the architect of the NeXTStep Mach operating system. He impressed me as a sweet, got-it-together kind of guy. I would have loved working for him. But they never called me back and I never called them because I couldn't see leaving my wonderful cocoon of Santa Cruz and resuming that commute to Redwood City, of which I was well familiar after 4 years of Santa Cruz-to-Sunnyvale-and-back. Makes new cars old very quickly.

I'm writing this because last night, when I inadvertently brought up my browser to cnn.com and saw the news, I was devastated. Shockingly devastated. Yeah, I have a few cute stories about Jobs, and no, there was no way I would have enjoyed working for him given the butterfly nature of my love for software development.

But, as I was telling a colleague at SAMBA this morning, like me, Jobs was an adoptee. Of course, he was also a fellow graduate of Homestead High and our silly allegiances to famous people feel quite real. But way back in the 80's when he became famous, I looked up to him as a fellow adoptee because I loved how he handled it. It gave being an adoptee real respect. He loved his parents and he didn't let the adoption thing hinder his life one bit. At the time, I thought it was kind of weird that I didn't have the slightest inclination to find out who my real parents were. My harsh self-appraisal was accented by what I'd been seeing on television at the time. Groups of adoptees were easy to find on day time talk shows, whining about how incomplete their lives were having no knowledge of their "real parents" and were confronting the fact that the adoption process was somehow overly secretive. It was strange to me that, in the same reaction, I was both repelled by these people and somewhat guilty that I didn't share their protest.

Jobs made me feel like I was normal. Better than that, he legitimized some of the crazy things I'd done along my career path. His unintended stamp of approval on how you can lead an adoptee's life without driving yourself nuts and, in fact, turn it into an outlaw'ishly productive and inspired life, was important to me. I had no idea how important his inspiration was, or how deeply I had internalized that message, until my reaction last night.

Thank you Steve,
David

Thursday, September 29, 2011

CNBC's "Explain This" makes KhanAcademy a Pioneer of the "HyperSchool"

Anybody think that CNBC has delivered the latest unanticipated Internet service that's a (revolutionary) turn for the better?

If you peruse my blog, you know I'm a big fan of khanacademy.org and Sail Khan's library of over 2,000 9-to-15 minutes classes. Perfect for the Web, for our rapid paced lifestyle and increasingly limited attention spans.

Why do I think this new features turns khanacademy into a new class of school I'm going to call "hyperschools"? Because it is the beginning of the destruction of our view of education as a silo where school is school and it happens during school. With "CNBC Explains", CNBC and Khan are showing that learning can be more spontaneous, more targeted, more convenient and more "in the moment."

The parallel is with hyperlinks themselves. They give the Web its ability to cross-wire information. Following hyperlinks, you can follow all kinds of trails of content (knowledge). Hyperschools, of which there is now only one -- khanacademy.org --, provide the same flexibility. Let me educate myself by attempting to read this interesting article about LIBOR (inter-bank trading rates) and I'll click on these "Explain This" icons as I see fit in order to more deeply understand what I'm reading (by following a quick 10 minute Khan class focused on specified, relevant topic (to the article).

Another metaphor is the Star Trek-like scenario of "I want to be expert enough to follow this article on Treasury Bonds, so I'll take a pill on everything I need to know to understand the article.

Hopefully, you get the idea. It feels like a subtle new B2B enhancement of a company's web site (CNBC to Khan)... but the more I thought about it, the more I like the feeling I got of yet more breaking of old barriers to education. Love it!


Monday, September 19, 2011

A more reasonable way to comment blocks of shell code...

I love using shell script for launching java and groovy apps because they're so good at setting the table in a way that keeps the application much simpler... especially when the shell script can handle common needs which is often the case for Operations-style functionality. Example shell script functionality includes

  • determining if there's sufficient space on the file system,
  • collecting files to process,
  • configuring the environment (as in defining traps that clean up temp files and remove lock files when an app closes either naturally or via control-C interrupt, or re-directing I/O so that a control-C won't kill the process you're running).
Because operations-style applications suffer from out-of-site, out-of-mind syndrome, having a solid strategy with shell scripts that can manage and measure the operating environment, and scream bloody murder when things aren't right, makes them a worthy design component.

I've never liked "the fact" that to comment out a block of code in unix shell programming I had to insert # symbols in front of every line.

I found out over the weekend the ideal way to do it using a here document and the ":" operator, which is a no-op

# all this code inside this section document
# is now invisible to the shell interpreter
# Add to use a # anywhere.

And here's how to do the equivalent with a here document.

: <<STUFF_TO_PASS_TO_COLON
all this code inside this here document
is now invisible to the shell interpreter
Didn't have to use a # anywhere.
STUFF_TO_PASS_TO_COLON