Data Quality : Concepts, Methodologies and Techniques
by Carlo Batini and Monica Scannapieco
2006 Springer

Rating:

This semester I had the unpleasant experience of being told by the campus bookstore, the day classes started, that the book I intended to use for my class was no longer in print and used copies were not available. I originally geared the course towards design philosophy and incorporated the book, Software Design, by David Budgen, an excellent book and its unavailability is incredibly frustrating.

As I was unable to quickly find a suitable replacement, I decided to take the course in a slightly different direction and find ways to discuss in more depth than I had before the notion of “quality”. To this end, I found Data Quality. After working through the text, I found it to be broad enough for incorporation into the class and serve as a starting point for deeper discussions but it also was unexpectedly dense, a bit discontinuous and, probably the most important, appears to contain significant errors which is why I could only rate it 3.5.

I still recommend the book as I think it contains good material but it’s important to know what to expect.

The strength of book lies in its content and to some degree the high-level organization of the chapters. I believe it works well as an undergraduate text (senior-level course). While the students may find the details challenging, getting a basic understanding of the why data quality is important, how to establish and implement a data quality framework and how to both handle the integration of different data sources and predict the quality of the end integration from the quality of the data sources is readily accessible.

The book provides plenty of references which enables deeper discussions of the topics presented. The survey-style approach the book uses provides the opportunity to direct students to follow through by locating and reading the papers/texts in which the original model was defined. In fact, to truly understand some of the topics in detail, further reading is a must. In essence, the book provides a good point of departure.

The survey style also means that there is limited attention to examples (though they are present). I found it useful, as an assignment, for the students to develop their own examples to illustrate the concepts. While this works when integrating the text in to a course, for someone reading this independently, the lack of solid examples could prove difficult. I don’t particularly see this as a negative given the point of the book.

In terms of minor weaknesses, more consistency with the examples (and to some extent better examples) and the use of unnecessary technical details are two areas that detract from the book’s quality. In terms of illustrating the former, consider the quality composition discussion in section 4.2. Part of the difficulty in understanding the topic is that multiple models and techniques were used to illustrate compositions in different dimensions. On one hand, it might be unavoidable since not every model adequately handles all dimensions. However, it was difficult to keep track of what model was being applied in what situation. Perhaps with greater clarity when establishing the context the discussion would be easier to follow. But I think focusing attention on one model at a time would simplify the delivery and help the reader see the application.

For the other case, there are often instances where overly fine details lend themselves more to confusion than to illumination. Section 5.4.3 contains descriptions of several comparison functions. In the case of n-grams, the Jaro Algorithm and Token Frequency-Inverse Document Frequency, the inclusion of explicit formula with the description is rather confusing as they are presented with limited context. For a reader who is not well-versed in a given approach, I would argue that it detracts from the point of discussion. It would be much simpler to just describe the approach as none seem to be explicitly invoked at a later time (or at least I don’t recall their use). It would be much more reader-friendly to provide a reference. In fact, in that same section, this approach was used for the description of Smith-Waterman, albeit without a reference.

The most difficult aspect of the book are the errors. I use the term “error” in the sense that, after much consideration and further research, I was not able to arrive at the conclusions presented by the authors. I fully understand the error may be on my part at which point I would concede that the issue would reflect the authors’ lack of clarity for a given issue. Errors range from something a little more involved than a simple typo, which a reader could with a little thought reconcile, to errors that lead to fundamental confusion. An example of the former is at the top of page 82, where |r 1 | should be |ref(r 1 )| (and sim. for ” r 2“). A more fundamental problem is in the description and interpretation of the of Figure 2.5, where I believe the area Cb should be the area bounded between the line, y=c_max, and the curve, C(t).

As stated earlier, I think the strengths of the book outweigh the negatives and would hope that others would not use the weaknesses pointed out here as a reason to avoid it. In terms of a class text, I think it illustrates to the students that print can’t be taken at face value and active reading is always a must. Plus, it affords the opportunity to direct students to correct error and/or explain the confusion. (But do note that I am not advocating for this style to be adopted by future authors!)

1. Should I pursue a certification in X?
2. Why would I pursue a degree from a university as opposed to a tech school?

With respect to the first questions, a recent article by Warren Wyrostek on InformIT discusses issues with certifications. The problems span from the simple mechanics – vendor-centricity, inability for certification processes to truly measure competence and continuous renewals on the part of the certified – through more economic reasons for why the certification bodies will struggle.

The bottom line is that a certification is worth pursuing if it is immediately relevant to a specific job position for which you want to apply. Of course, this gives rise to the question of how often is it relevant. To understand this some I did a few searches on Monster.com (date: 2008-03-19) in the categories of computer services/hardware/software, internet/e-commerce and IT/software development with no region specified :

• Searching for “ccna” yielded 1021 jobs.
• Searching for “MSCE” yielded
• Searching for “a+” yielded 1114
• Specifying no keywords resulted in 5000+ jobs

After filtering the overlap by searching for positions requiring two or all three of the certifications, the count for such positions involving at lease one of the certifications was 1115. (Searching for “ccna msce” resulted in 22 position, “ccna a+” resulted in 134, “msce a+” resulted in 14 and “ccna msce a+” resulted in 3.)

I also examined random positions in each search to get a feel for whether the certification was desired or preferred. I wasn’t able to nail down a sense of percentage by visual inspection (there were too many jobs) and trying to use the search to discriminate between the two proved difficult since there are various ways to state that certification is required. It’s worth noting too that the jobs requiring A+ certification were primarily entry level help desk positions and something most graduates of a university would not pursue (though they may take such a position if other opportunities were not available to them).

I address the second question roughly by explaining that the program offered at Penn State provides a considerably broader set of experiences with one of the primary goals being graduates that are adept at critical thinking, are adaptable, operate will in the presence of significant ambiguity, can pull from a diverse exposure and able to be life-long learners. Technical schools tend to focus on a specialized set of skills over a specialized set of environments which may become obsolete over time.

While graduates of a tech school are able to find employment, they often can only advance so far. Organizations expect staff in the upper level positions to have obtained degrees at reputable universities or colleges. Though I don’t know for certain, the rationale is probably based on the intent of the university education. They want some sense of measure that the employees in positions making critical decisions for the organization are prepared for such tasks.

I’m still trying to organize my thoughts on this latter question. There are many nuances that require further thinking. I’ll get back to it before long.

What concerns me more is the reaction of several (perhaps many but I didn’t search long enough) that identified the situation as a revolution, a changing tide of the way conferences are executed. In an article on CNNMoney.com, Dan Fost used the term “Conference 2.0″. This is probably the start of a buzzword war and we at large will be the casualties. But more importantly, I am concerned the event will spark a notion that this is the way of the future and conference organizers should heed the call. I haven’t seen much discussion regarding this or similar concerns and I hope that eventually more sources will offer balance to the topic.

My primary concern stems from a “proof by example” approach. This isn’t necessarily suggesting that the authors of articles relating the amazing influence of mob force on an organized event but more in the interpretation of the masses thinking that they are welcome to drive any public discussion in any direction that pleases them. Such behavior would become rather distracting. I can envision many good presentations being derailed by a group thinking they have the vision of the people.

Certain venues are more conducive to real-time attendee-defined content; for example, technical expos. Expos are generally driven by industry elements demonstrating their latest wares. These are typically pre-defined by marketing with hired expert addresses and vendors who pay premiums to be present. The price of admission for the end user is steep as well (but you will get your fill of free food and drinks).

Many of the talks at expos are by paid experts who travel the circuit giving the same talk many times. They are focused on their topics and typically have a great deal of experience. Anecdotally, from my experience, many of these experts are attached to some business or other, be it a tool vendor or consulting firm. They typically are there to sell their ideas as much as they are to present them. Since the talk is likely to be similar to the last one the attendees heard from the speaker (as has been my experience), this type of venue might lend itself well to an audience-driven format especially if there is a control on the way the audience provides direction.

The reason that this may work is that the talks provided at expos are often directed toward the average user (of a product or method), or perhaps the advanced user, and not the expert. Many of the expo talks I have attended where I was more on the expert end of the scale were entertaining but I can’t say they were all that useful for inspiring something new. The average/advanced users have specific questions they want solved and in a venue where they may feel they have paid for a service, they want to be able to ask them and not necessarily hear the party line.

On the other end of the spectrum would be an academic conference. In this venue, the content is necessarily not something to be determined in real-time. A presentation at an academic conference is, ideally, one that presents some advance of knowledge in a specific area. It’s something that may only be known or understandable by a small group of people, perhaps just the presenter. In this case the result requires a build up of context and arguments and derailing the presenter would render the talk pointless as the end goal may never be reached. Here, the Q&A would have to wait to the end.

This doesn’t mean that the those attending do not have influence over the content, it is just that the participation happens prior to the event. Conference committees may define the theme and topics for a conference but it is through a collection of reviews, often done by those intending to attend the conference, that give rise to the final content.

If you were to argue that Zuckerberg’s interview and a presentation by an expert at an academic conference aren’t the same, you would certainly be right. My point is that many won’t make the distinction and without a balanced discussion there will be some instances very soon where good events turn bad. Ideally the event will force event organizers to include more flexible agendas where it makes sense. Even academic conferences need a breeze of fresh air at times but too much wind makes it difficult to be in the park.

Embedding a messaging window is rather simple. To do so, you just need to change your Web Settings for your Windows Live account. Once you agree to expose your Messenger status, you can then select one of three embedding choices: full messaging client, a button or just an icon. The details are available from Microsoft but it pretty much just a matter of copying the generated code and pasting it into your web page.

Once you have the Messenger client embedded, there is still one more configuration needed before you can start receiving messages. I point this out because the setting is not mention on the Windows Live page and took me a while to find. Specifically, on your messenger client options, in the “Privacy” section, you need to uncheck the box for “Only people on my Allow List can see my status and send me messages”.

I typically use Trillian and this setting could not be set from that client (or at least I couldn’t figure out how to do so). When I first started playing with the embedded client, I wasn’t receiving messages. It took me the better part of a day to realize the problem and figure out the solution. I have been having trouble with Trillian recently and may have to abandon it.

Now, let’s take a look at the details.

Note: This may be a bit of a mess. The idea is there but it probably needs some polishing. At some point I plan to write a tutorial on how to do more with these APIs but I wanted to start with a simple overview.

The three options provide generated code that is easy to include. The Messenger client is pretty much WYSIWYG but you can configure the way it looks via the IM Control API. The status of the target user (the one who will be receiving the messages) is indicated in the Messenger client.

If the full client is too large to include in your web page, you can still indicate the status of the target user and provide a link to a Messenger client (a web page containing the client, perhaps just the client itself). The status is obtained using the Presence API . It provides access to the target user’s Messenger status as well as access to the standard Messenger icons (these can be overridden).

I think the easiest way to get started is to use the generated code for the button:

I omitted to generated URL's. They are long and distract from the discussion. Generate your own code and examine them.

The first <script> tag references the source code file which contains the function to be used as a callback. The callback is registered in the second <script> tag. The generated code provides both of these. The second one you will want to leave as. The first is where you can customize the way the target user's presence is visualized.

The callback you provide needs to accept a presence object. There, you can create any HTML you desire and splice it into the document. When the second <script> tag is processed by the browser, your callback will be executed.

To start customizing, take the generated code and download the callback code provided by Microsoft. You can access it by opening the URL specified in the src attribute of the first <script> tag. Once you download it, place it somewhere accessible by your development page and modify it as desired. Be sure to update the src attribute as well.

It's worth noting that in order to obtain presence and create a Messenger client, you need to have the user id associated to the account for the target user. When you create the HTML code for a client, button or icon, you can find your account id in the code. It should be obvious but, if not, you can find it at the end of the string value associated to the id attribute. (It should be 16 characters long.)

Learning Java, 3e
by Patrick Niemeyer and Jonathan Knudsen
2005 O’Reilly Media

While looking for a different text to use in IST 240, I found the text Learning Java by Niemeyer and Knudsen. This book has done nothing but impress me. While in the end it might make for a difficult text to begin programming in Java for the lesser experienced, as many O’Reilly texts can be, it is going to be a solid addition to the bookshelf of an aspiring information technologist. Hopefully this title will continue to evolve.

For a few semesters, I had been using Java in 60 Minutes a Day by Rich Raposa. That text was well received by the students and the content spanned the core material I needed for the class as well as advanced topics (Swing, network programming, JavaBeans, database programming) that interested students could engage if they so desired. Unfortunately, the book has become significantly dated. It is based on Java 1.4 and so is missing topics introduced in Java 1.5, generics being one of more important topics.

I contacted the publisher about a year ago (maybe more) to inquire as to the future of the Raposa title. They replied and said the intention is to not continue it. I explained my interest and support of the book, and even offered to help update the contents, but they had no interest.

Last semester, I used Hebert Schildt’s, Java: A Beginner’s Guide, 4e. For use in IST 240 (and probably as a good book for learning Java), there were a few problems. One of the more important ones was the lack of any treatment of the Java Collections Framework. Unfortunately, I didn’t realize this until we were covering the Collections Framework in class.

What is stranger still is that Schildt did include a chapter on Java Generics. Why generics are discussed but the Collections Framework omitted, I will never understand. Most who venture to learn Java will be more apt to use the Collections Framework more than they would define their own generic classes. For a good starting reference this seems problematic. It did have include chapters on Swing and multithreaded programming but such topics were thinner than what Raposa had provided. The student opinion of the Schildt book was low as well. They found aspects of it sufficient but expressed that there was room for much improvement.

Fresh from the disappointment of Schildt, I more carefully examined the contents of Niemeyer and Knudsen. I noted that it covered all of the topics I planned to include in IST 240 (or that students would need). It covered the topics Raposa had with the exception of database programming. However, it also included chapters on processing images, audio and the Java Media Framework as well as more advanced text processing. Probably the more crucial inclusion for the course, though, was the chapter on Java and XML. The treatment of XML in the text was sufficient enough, perhaps supplemented by online readings, that I felt comfortable eliminating the XML text that I usually include.

More than the inclusion of particular topics is the degree to which the authors explain the topics. It is more than a casual treatment. There are many in-depth explanations and discussions of interesting language features and obscure library elements (or at least those typically used by the public). For example, a short section in Chapter 16 discussed the java.awt.Robot class; and the Scanner and Matcher classes discussed in the text processing chapter, Chapter 10.

From an experienced user’s point of view, I have consulted the text several times on miscellaneous topics and found that the authors provide deeper insight than usually included in a book with a title suggesting the reader will “learn” a topic. My bias is to assume that such types of titles imply a something suited for a beginner but not one with experience. This book presents itself well to both crowds.

It might be argued that the book’s inclusion of such topics render it intimidating or difficult for the novice. There might be some truth to that. Perhaps for an introductory programming course, this text would be a little heavy. However, for a course such as IST 240 where general competence with programming is assumed, the book seems to be a good choice. Students might have to read passages twice or consult with the instructor for further clarification (all things we want students to do anyway) it will be a while before the needs of such readers will outgrow what the text has to offer.