Wot no e-mail?

On this day when the Entente Cordiale (sp?) definitely took a hit on the chin let me do my bit for Anglo - French relations by praising a French leader. No not sulky Sarky - who more than likely won't survive his bid for re-election in 2012 - but Thierry Breton, CEO of Atos, who has decided to phase out internal e-mail over the next 18 months. Inspired leadership.

Cheap at half the price

Here at the Depatment of Hopes and Dreams we're having a lovely spat with one of our software vendors at present. The issue is that my boss is questioning the need to pay a quarter of a million dollars in annual maintenance fees when he believes we only use about $40,000 worth of the product.

The interesting thing is that the software supplier has recalcuted the maintenance bill twice now using different breakdown structures. Both breakdowns don't help clarify exactly what it is that we're paying for. In fact the only figure that remains the same is the invoice amount - $250,000. Funny that.

The wider question is that once you've implemented something and it is successfully bedded in why bother paying maintenance ever again? The software vendor will say that you're our of support and that you original software licence will be revoked but if you do the cost benefit anaylsis you might find witholding the maintenance over say 5 years might pay for the replacement software further down the line. By that time the software might be half the price anyway. It's certainly worth some consideration.

How much should you pay for software?

I've always had a problem paying lots of money for software, which is odd when you think about it. I'll happily pay for hardware, or to see a movie or to listen to a CD. In fact, the only software I think I pay without much hesitation is a computer game.

Out of the twenty or so apps I've downloaded onto my iPhone the only paid one is the very successful game Flight Control which cost me the princely sum of $1.19.

So at Oracle I was working with a sales rep who was trying to land a $10m deal with a large australian multinational company for a global license I was surpised, given that this account made up about one third of his territory, that he was chasing a deal that would probably limit what he could sell in future years.

We had a discussion about what software was worth and it was illuminating to me. As he pointed out - to him the software was worth the cost of the CD - a few cents and nothing more.

The funny thing is that a few years earlier I worked for a mainframe software house who when the annual results we're due and the numbers looked bad had a dodgy practice of cutting a few tapes bunging them in a storage cupboard and reporting the software as millions worth of assets. (They were later censured by the Stock Exchange for this practice).

I guess the old marketing saying is true - something is only worth what somebody else is prepared to pay for it.

But you can polish a Turd!

A old friend of mine frequently uses the phrase "You can't polish a turd". Until this week I believed this to be true until today.

We're four years and $140m into a BPR project that has a major SAP implementation at its heart. The intesting thing is that ten days after Phase 1 go live, and 1,500 related service desk calls later, the message I'm getting in the crucial run up to the end of financial year is that large parts of Finance simply can no longer do their day to day jobs.

Funny that because the SAP Implementation Program Manager's status email today doesn't really correlate with what I'm hearing on the ground.

Oh well. I suppose someone's just found that special way of polishing.

Keep it Simple Stupid or Keep it Stupid Simpleton

There's an urban myth that says NASA spend millions designing a pen that would work in space whilst the Russians got by with pencils. This is a perfect example of KISS. In reality the pen was developed cheaply by an independent contractor and sold to both NASA and the Soviet Space Agency.This myth and its ilk often used to be quoted when comparing the Cold War forces pitted against each other. By all accounts the MiG 29 is very low tech when compared to the F-15 Eagle for example and there was a time in the 80's when supposedly F-15's were only serviceable every 4 days out of 10 because of their inherent complexity.

This phenomenon has parallels with some of the choices we face in modern day IT. During the internet boom I was engaged bya dotcom to define a server architecture for their new revised classifieds website. The brief was that it needed to be scalable and have a very high availability. I provided the client with three options at the time which were:

- simple - a single highly resilient server with no failover capacity
- moderate - a hardware based failover cluster which required a hardware reboot to failover between nodes
- complex - a hardware/software based load balanced cluster which could handle a gracious soft failover

Being accustomed to having to cost justify these options I fully expected the client to select the simple or moderate solution. To my utter surprise they didn't blink an eyelid as went for the most expensive and complex option. They perceieved that the risk of an unplanned outage was so serious that they would literally spend millions to guard against the risk. They also explained that the sort of kudos they would get in the industry by following the likes of Amazon and announcing an infrastructure purchase in excess of a million dollars would enhance their perceived market valuation pre IPO.

In hindsight, I wish I had never proposed the complex option or that I should have been more vociferous in opposing it. Previously I'd had great success in applying the moderately complex model at previous sites. However, these were heady times and so long as the client understood the risks I was prepared to give it a go. The result, despite all my best efforts, was a disaster. We had severe performance issues from day one and our reliability was worse than it would have been had we picked either of the simpler options. It was humbling for me and a great reminder to me to always apply KISS.

The interesting thing though is that if we don't ever try to stretch ourselves now and again we'd all be building Mig 29's. For the record the F-15 supposedly has a 104 to 0 kill ratio.

Madness in the Method

In the UK there was a famous 80's TV commercial for British Telecom starring Maureen Lipman (http://centuryads.blogspot.com/2007/01/you-got-ology-1987-launch-of-bt-beattie.html). She plays a stereotypical Jewish grandmother who upon discovery that her gransdon has flunked all his exams except Pottery and Sociology replies "He gets an ology and he says he's failed... you get an ology you're a scientist." Maybe, just maybe, Andrew (the grandson) didn't become a scientist but got another ology ... a methodology instead.

This would tie in with how many methodologies are used (and abused?) in today's IT. From what I seen methodologies are used as a crutch by below standard IT consulting organisations/individuals to present a veneer of professional competency and mask the lack of any real ability.

Don't get me wrong, there's nothing inherently wrong with a good methodology as far as they go, but don't for one minute believe that an ology will somehow replace common sense, experience and good judgement.

Six Degrees of Preparation

Over a coffee the other day with a friend we were discussing what qualifications were most appropriate for a 21st Century IT professional. Obviously you would assume that a BSc in Computing or equivalent would rank highly, maybe even a BA in Information Technology, although at my Uni this were regarded as a soft option in IT degrees. Other studies related to New Media and Communications may also be relevant.

(FYI I studied on a Combined Sciences course of which one of my majors was Computer Sciences).

The funny thing is that when I got my first job in IT I was already COBOL trained so productive from pretty much day one. Not so for a couple of my new starter colleagues who read History and English Uni degree respectively. They embarked on a 6 month programming course.

IT is pretty unique in that way as it has alway been open to all comers. This is something that would be incoceivable in most other professions unless you had a relevant degree (i.e. Law, Medicine and Engineering) or even new fields like Biotechnology.

Back to the original question - what is the best degree. Well my mate believed that it would be a Law degree, followed by an MBA, because as we to oursourcing and cloud computing he believes that the only relevant skills for modern IT was drawing up and managing contracts.

That's a real shame.

IT Nirvana

Over the last few posts I have raised the unfulfilled concept that was prevalent in the early days of Open Systems - namely the Corporate or Enterprise Database. This concept was where corporate data would be stored and accessed centrally from a central managed place. Of course the central database never happened and there are lots of reasons for this, but I like to think of the Corporate Database as a kind of IT Nirvana. Twenty years ago if we could show IT managers what their systems landscapes would look like today I think this idea may have taken off and just possibly we could have a achieved that blissful IT state.

Instead we face the following issues:

- Systems complexity - get your local IT architect to print out your current systems landscape. Even better get him to walk through all your interfaces.
- Data Warehouses - Often the holy grail of the data warehouse is the search for a single view of the truth. This is because your multiple systems will all have different versions. If the data was kept in one place then we wouldn't need to build expensive and complex data warehouses.
- Master Data Management - The Corporate Database is the MD solution
- Enterprise Data Bus - Why would we need a bus if all the data sits in the same location?

So are we likely to find spiritual enlightenment in IT soon? Well not so long as we can't see past the next quarter, or FY budget for that matter.

Luddites Unite

When I first started in IT I was a COBOL programmer and the database I mainly used at that time was IDMSX. We had a large powerfull mainframe that supported over 250 transaction processing applications. The interesting thing was that at a conference my boss got talking to his equivalent from a similar sized organisation. They had the same mix of systems and same sort of user community - only their mainframe had less than half the grunt of ours.

Why? Well they had rejected database technology and kept everything in flat files. I can't vouch for how responsive they were as an IT team but it makes you think.

This highlights something we've all seen happen in IT over time - huge advances in chip, bus, memory, disk and network technology are soaked up by software bloat. OK this is a simplistic argument and in reality corporate IT is doing a lot more than it did 20 years ago, but it still makes me wonder just how those flatfile systems would perform on todays equivalent platforms. I think they'd scream along.

Suspect Packages

In the 70's and 80's companies used to write bespoke software for all their needs. Everything from payroll to financials to core business systems would be written inhouse in COBOL, RPG, PL/1, Pascal and the like. Then in the 80's and 90's the concept of packaged applications became popular. The rationale was that it was better to purchase and implement off-the-shelf packages rather than develop from scratch. This made perfect sense and much of my early IT work was involved in package implementation. Ideally a package could be implemented in months rather than written in years.

Software selection became vital as the business would often need to assess how flexible that package was and how easily it would fit into the existing processes. Working on the Technical Pre-Sales side it was always important to try to get a product fit of 90% or better. Inevitably though there was never a 100% product fit and in these situations there were two potential solutions:

1) For the business to adjust its way of working (sometimes called Business Process Reengineering)
2) For the package to be changed or modified (i.e. mods)

In the vast majority of cases the latter option was chosen as the business was reluctant to take on change. In one scenario I worked on a customer site where 70% of the programs had been modded making you wonder why they had ever selected the package.

The one exception to this model has always been SAP. They have always insisted that BPR is a fundamental principle of their implementations and as far as I know SAP implementations are never quick. Here at the Department of Hopes and Dreams we're currently 3 years into a SAP implementation and the frustration it is causing the business is unprecedented. Makes me wonder whether the advantages of packaged apps are still there.

Master Data Blues

Master Data Management is currently one of the hot topics doing the rounds of Enterprise IT. For those unaware of the concept MDM is basically the storing, either physically or virtually, of all corporate reference data in a single MDM repository. Consultancies love to sell MDM solutions for two reasons:

1) They are quite easy to justify
2) MDM technology, whilst expensive, is available

So in theory we should all be swimming in MDM's by now. I don't know about you but I'm not. In fact a friend of mine became the first and only person I know to land a successful MDM Project.

So if we agree that it's the correct thing to do, and that the technology is available then why don't we see MDM's all around.

Simple really - the business side of MDM is very, very hard so before you begin you'd best make sure that the business understand what they are biting off.

Of course, if we'd kept a tight rein on our corporate IT systems in the first place then we wouldn't need MDM would we!

Home or Away

In the previous post I touched on working from home. Many of us have PC's at home or work laptops on our desks so in theory many of us could be doing the same job from home. In the main, however, we don't and I believe that there are a number of reasons for this:

- Communication - I think face to face communication is great and by working from home we'd lose this. However, as I've already touched on in "The Great Lost Art of Communication" I think this is becoming less relevant in today's workplace. If we're emailing each other arcoss the office floor we might as well be emailing each other across the city.
- Management vision - I don't for one moment think our management have seriously sat down and debated whether they should contemplate offering work from home options for large parts of their workforce.
- Measuring how we work - Easy enough for Sales guys and call centre workers but what about the rest of us. Performance is usually measured on hours and perception of our abilities, etc. at our annual reviews. These can be difficult to assess when you work from home. If we effectively want to measure how our home-based workers are doing then we would need to put a lot more effort into assessing how long tasks should take. Perhaps we even need to a system of paying not based upon hours but based upon achievements!
- Trust - If we still measure by hours then this brings the question of trust into the equation. Often work from home is abused.

None of the above is insurmountable, but they do require management vision and an ability to think outside the current way in which we allocated work and measure our success in achieving our goals.

Congestion Blues

My average work commute speed is about 8mph*, which is just better than the average London rushour car journey at 7mph - worse than the horse and cart achieved at the start of the 20th Century.

Cars are great so what's went wrong? Well the problem is the tarmac and the fact that we all want to use the same bit at the same time. So if that's the problem then what are the possible answers?

- Build more roads - yet studies have shown that new roads encourage people off public transport and into cars with no net benefit to congestion.
- Build great public transport - yet most governments are too bankrupt to fund the sort of public works programs required to make major infrastructure improvements here.
- Work from home - the best commute is the one you don't have to take. With boadband and mobile technology we have the infrastructure and tools that enables us to really embrace this solution, but so far it's only a minority solution.

So what's this got to do with IT? The answer is below:

- Build more roads - is the equivalent of building more interfaces between our systems. Except that in our case each road is made from different materials, using different construction techniques, different traffic lights and different road signs.
- Better public transport - is the equivalent of building an enterise bus architecture. A great potential solution but still very costly and complex.
- Work from home - back in my earlier post "Redundancy Blues" I lamented the lack of the single corporate database. Imagine if all the data our company held was in one database. We would ensure that it ran on the best hardware, backup and recovery would be easy, we wouldn't need to build data warehouses to try to discover a single version of the truth. In short,if we had realised the potential benefits offered to us by Open Systems we'd be in a far better place than we are today.

Oh, well, must go and get my pick axe. I'm building a new road today.

*Please note that I consider myself very lucky that my commute is only half an hour. Across the world many, many people spend about 2+ hours per day commuting to work and back - that's about 25% of the time that they are actually there!

Avatar and the Real Time Data Warehouse

Last night I saw a feature on the making of Avatar. What caught my attention about this was the way in which they had created a 'virtual camera' through which James Cameron could observe the live action of his actors morphed into their avatar bodies and displayed in the CGI generated world of Pandora - in REAL-TIME.

In my business world, which is about as far removed from Pandora as you could get, I build data warehouses and have been doing so for many a year. From time to time the concept of Real-Time Data Warehousing has arisen and largely been dismissed for the following technical reasons:

- Data Dependencies
- Slowly Changing Dimenions
- Complex Transformations
- Updating Aggregate/Summary Data and Cubes

All of the above are still valid, and there would still need to be a valid business reason to go to the extra cost and expense of building a real-time data warehouse as opposed to a cheaper batch one.

However, I'm sure the technical hurdles that the Avatar movie makers overcame must have been a whole lot longer than my sorry list. Maybe it's time for a rethink!

The Price of Everything, The Value of Nothing

Back in the days when I wore a proferssional services hat the T&M (time and materials) contract was prevalent. T&M was good for a consultancy as it loaded most of the the risk on the client. I also think it did a good job of focussing the client minds as the last thing they wanted was expensive consultants sitting idle whilst they provaracated over making a decision.

Over time clients more often moved to a Fixed Price model for their consulting engagements which helped control their costs. It also shifted the risk to the consulting firm and for that I know that my consultancy would add a margin of 30% to offset that risk. What the clients failed to understand in the shift from T&M to Fixed Price was how that would change the mindset of the consultancies that they engaged.

Back in the T&M world we really focussed on doing things right. We knew we were expensive and we knew we had deadlinesto meet. I can honestly say that when I worked on site I believe I acted in the clients best interests. The problem with Fixed Price is that the consultancy focusses on doing as little as possible to cover its contractual obligations.

Hence the client may think they're getting a better deal by fixing the price - but you know what - I'm pretty sure they haven't figured out how to measure whether they're getting good value or not.

The MobCon War

A little over a month ago in my post 'The IT Wars' I decried the lack of a current IT War. Like it or not wars are the engine of innovation hence the old quote that in 300 years of peace the Swiss invented the Cuckoo Clock whilst in the Second World War the protagonists invented the Radar, Rockets, the Jet engine and the Atom bomb.

My earlier assessment was that the following IT Wars had been fought and the victors had prospered.

The desktop wars: Microsoft Windows vs IBM OS/2
The server wars: Mainframe vs midrange
The RDBMS wars: Oracle vs Ingres
The browser wars: Internet Explorer vs Netscape
The Search Engine Wars: Google vs Yahoo

In my defence the focus of my original post was commercial IT but rereading the post in light of the second reading, and also that fact that on occasion my blog has strayed into the MobCon space I figured it was time to put the record straight and comment on the current war at hand.

To paraphrase the Excapite blog: The MobCon Wars. If you want to understand the MobCon I'd recommend some essential reading at http://excapite.wordpress.com/

There are lots of protagonists originating from the technology, telcos and media fields but I believe that this war will be fought most bitterly between Apple and Google. Up until a couple of years ago these two tech giants seemed to be peacefully co-existing and focussed more on attacking Redmond. Eric Schmidt even had a seat at the Apple high table. There are even rumours that the two companies has a no poaching of staff agreement.

Judging by the recent second hand quotes attributed to Steve Jobs in a recent Town Hall meeting it would seem that relations are somewhat strained between the two tech goliaths. He appears to be somewhat aggrieved that the Mountain View mob have strayed onto his turf with the Nexus One and potential Chrome based tablet whilst Apple to date have stayed out of the Search Engine field. Nothing like a bit of siege mentality to get the minds focussed and steel your troops for combat.

You can place your bets on the eventual winner but if I was a betting man I'd have to put my money on Apple. After all they're a company I choose to spend my hard earned cash with. When's the last time you bought anything from Google?

My Online Doppelgänger

It seems that a few months after I started my IT Journeyman blog that I have an online doppelgänger. That's OK because I'm not the jealous type.

Having skimmed said blog the following post http://www.itjourneyman.com/2010/01/16/data-warehouse-2nd-time-is-a-charm caught my interest and it's essentially a rehash of a few white papers on "pitfalls/mistakes to avoid when building data warehouses". The long and the short of the post is that your first data warehouse will be a failure but don't worry because the second one will learn from those lessons and succeed.

I love to say that this was true but imho its just not that simple. In my travels I've worked on first stab data warehouses that have been blinding successes and also third tries that have had no more luck than their predecessors.

There are lots of elements that go into making a data warehouse project succeed or fail and often the initial expectation setting exercise is crucial. We have to be very careful in determining the criteria of what makes a data warehouse work and what doesn't.

It's a bit like marriage and divorce. Most people would assume that definition a fifty year marriage must have succeeded - but what if the husband and wife were at each others throats for the duration. Likewise divorce after 10 years is seen as failure but what if you've produced a couple of wonderful and well adjusted kids and went your own way amicably. Expectation is everything.

What I can say is that in my experience Data Warehouse projects are difficult and that's why I choose to work in that field and not implemeting somebody elses off the shelf package.

Data Warehouse Projects are voyages of discovery and it's what we learn along the way and not necesarily where we end up that's really important. The problem is that most organisations and most PM's just don't understand that yet.

7x24

If you've been around in IT for a while you've probably come across the term 7x24 meaning 100% system uptime.

I was once employed in London by a Investment Bank as a DBA where we were developing a mission critical global options trading system. Luckily the data volumes were small, the servers and environments stable and I'd had plenty of time to work through a reliable hot standby failover solution with an excellent UNIX Sysadm. All was good in my world.

Then during the preparations for go-live the topic of Availability arose. The Project Manager threw into the mix that we had to guarantee 7x24 availability.

My response was that we could aim for 100% uptime excluding planned outages but that we couldn't guarantee it. This resulted in a bit of table-thumping, as was quite often in IT projects in an Investment Bank.

Suffice it to say that when I explained the costs and complexities involved in guaranteeing high that availability from a solutions side and the human side it the PM became a bit more reasonable, especially when I threw in the fact that neither Scott McNealy nor Larry Ellison could guarantee 100% uptime on the configuration of Solaris and Oracle that the solution was constructed.

So the lesson is that before you start discussing High Availability the metric that needs to be understood is the actual cost, either in dollars or reputation, to your business of the mission critical app being unavailable. Until you have that there's really no point in discussing the HA requirements of the system. The funny thing is that when I was consulting I designed lots of Technical Architectures and never once could I get that fact out of the client.

Assisting the Police with their inquiries

Back in 1998 I was doing some Pre-Sales Consulting for an Account Manager trying to sell a Data Warehouse solution to a local state police force. I badgered the salesman to let me use the above title as a tagline on the demo but unsurprisingly he didn't see the funny side.

During the demo the thorny question of Metadata came up. More precisely - Consolidated Metedata. As I'd just come off a project where I'd defined the Metadata Architecture and Solution I was well qualified to answer the query.

At the time we had three sources of metadata for our solution. These were:

- The Database Data Dictionary
- The CASE/Data Modeling tool in use
- The ROLAP Semantic Layer

Note that this we didn't use an ETL product that would have been a fourth source of Metadata.

Now the interesting thing here is that all the software was written by the same company in the same software labs so one would hope that some level of shared metadata would be possible. Alas no. Not only did the metadata in each repository overlap but there was no easy way of combining it into a single source of consolidated metadata repository.

I answered the question honestly that nobody had a good story here, not us nor our competition. I think the client appreciated my honesty here. The account manager obviously not wanting to leave a bad impression did what all account managers are prone to do and started promising vaporware with some cock and bull story about the software labs in California working on that problem.

The interesting thing is that here we are over a decade later and I've still to see a good answer to this problem.

Taking the Mountain to Mohammed

I've been working in the field of Data Warehousing for some 13 years now. Actually my first every data warehouse was a Reporting System I did back in 1992 long before I'd ever heard the terms DW & BI but that's another story.

The interesting thing that, so far, has been a constant in all that time, no matter what style of Data Warehouse (from full blown Inmon Corporate Information Factory to Kimball Federated Data Marts), is that we extract data from source systems and move it and load it into a data warehouse (be it an EDW, Data Mart, ODS, RDS, whatever). We'll use terminology like ETL, OLAP, ROLAP, Cubes, Star Schemas, Metadata, Slowly Changing Dimensions, etc. along the way to baffle the business and make ourselves seem clever but fundamentally any data warehouse or data mart involves moving data from a source system into target reporting system.

Back in the 90's this made perfect sense because it was inconceivable that we could slap resource consuming queries on reports against the mission critical core business systems.

Nowadays that just not the case. There are many technical solutions out there that could enable us to place a large and significant batch query and reporting load against our production data that would have zero impact on the core business systems. Technologies that spring to mind include Server Virtualisation, Disk Replication and Mirroring, O/S and Database Parallel Server technologies, etc.

The question is why don't we employ these technologies? I suspect that in the field of DW & BI we're in a stuck in a Kimball or Inmon rut and that for the time being we will continue to Take the Mountain to Mohammed.

Ah, but what about history I hear you ask? Well yes it's true that we often capture history in the data warehouse that we cannot keep in our online systems but often the need and justification for history is overstated. Besides another way in which we could keep all the history we'd ever need (and we probably already do this to some degree anyway) is to ensure that all PDF reports that are produced are kept online in some fashion. There are alternatives if we are creative.

Maybe within the decade well see a shift away from this and let Mohammed walk to the mountain for a change.