System Performance – Thinking Sideways

Share

I find system performance problems utterly engrossing. Although CPUs have accelerated (in line with Moore’s Law), and multiplied, and although memory has expanded, network communication speeds have increased and access times for storage media have shortened, software has become more sophisticated and we still find ourselves drumming our fingers whilst waiting for our systems to respond.

swirlyscreen

When you’re dealing with a performance problem you must think imaginatively and picture the way computing resources are being deployed. If you’re working on a client-server system, where large numbers of PCs might be accessing the same server software (often a database) it is more complicated still.

Sometimes a server processor is too slow (statistics might show that it is never idle), and processes must wait for its availability. In this case you must try to reduce the work it’s doing, by improving the efficiency of code, or by shifting processing from the server to the client.

Sometimes network communications are too slow, and you must reduce the volume of data being transferred between client and server, or reduce the frequency of transfer (web page size can matter in these circumstances, and program logic).

Sometimes a database is poorly designed and an application must ask too many times for data from too many records. This reveals itself as too much data storage activity, or too much database software processing. The answer, then, is to redesign the database, or configure it more effectively so that frequently accessed data are better cached.

Sometimes there is contention for resources. A large database system will often contain ‘control records’ that ensure consistent use of a system (such as a record holding the ‘next invoice number’, a record that many processes may need to update). If access to such records isn’t managed cleverly, one (or many) processes may have to wait for another process to finish its work. Sometimes the result is ‘deadlock’ when one process has ‘locked’ a record that another process wants, and that other process has ‘locked’ a record that the first one wants.

Instinct, and an ability to imagine these resources vying with each other, is a useful talent when it comes to solving problems. Doctors will attest that instinct plays an important role in the diagnosis of human sickness.

But sometimes you solve one problem and create another. Thirty years ago, when I was a junior programmer, I was working on the development and implementation of a financial management system that used a complex database to store its transactions. The client was exasperated by the poor performance of the system. Analysis showed that the CPU was idle most of the time, whilst the disk storage devices were overheating (metaphorically). I looked at the database design and I could see that it resulted in fragmented, almost randomly distributed data, when smooth physical distribution of sequential data was what was needed.

So I redesigned the database and built a caching mechanism that used an algorithm (using chains of pointers) to retain the most frequently accessed records in memory, where they could also be accessed without ‘locking’. Data access activity fell tenfold, but so busy was my algorithm (I mistakenly used bits rather than bytes as indicators in an unnecessarily sophisticated attempt  to save on program size) that the CPU started steaming (metaphorically) instead. The CPU simply couldn’t get through its work. I’d improved overall performance slightly, but insufficiently, and the client abandoned the system. I’d moved the bottleneck from one location to another. There’s always a bottleneck.

But perhaps the most fascinating and frustrating performance problem I was involved in (on behalf of LLP Group) was at Otopeni Airport in Bucharest. Our client, Air BP, was using SunSystems for its financial accounting and reporting (for both local and corporate purposes). They called us, stumped, because, from time to time, and unpredictably, the system slowed markedly and became too frustrating to use. It was a typical client-server system. There were about eight PCs in regular use, and a server sitting in the corner of the office that contained the SQL database that SunSystems used.

We spent hours at Air BP’s airport office. We played around with parameters on their PCs and on the server, and when we were there (as so often happens) the system worked perfectly. It was the exact opposite of the ‘demonstration effect’ which causes software to go wrong just when you want to show it to someone. In this case, it never went wrong when they wanted to show us the problem.

We were stumped too. From time to time we would put our (metaphorical) spanners down and sit and think about it, and have a cup of tea. And usually whilst we were thinking and sipping the system would start to slow down again. Was there a malicious child sitting inside it? We’d go and look at the server parameters, and, hey presto, it started working well again. We looked at the server statistics. The CPU wasn’t busy, the disk storage device wasn’t busy. There was plenty of memory.

And then one of my more logical or imaginative employees had a brainwave.

‘When we’re working on the server, it works well. But when we’re not, after five minutes or so it gets slower. What happens on the server when we’re not working on it?’

‘No idea.’

‘The screensaver switches on.’

And that was it. There was a lovely swirly screensaver that came on after five minutes of inactivity on the server console. So complex and lovely were its graphical calculations that it used 95% of the CPU’s capacity. We simply switched it off.

Sometimes it’s the last thing you would think of.

Advertisements

Beauty on a Small Scale

There’s a lovely new exhibition at the Hunt Kastner Gallery here in Prague of drawings, designs and photographs by Dalibor Chatrny (1925-2011), an influential artist and teacher who studied in Prague shortly after the Second World War, and who spent most of his career based in Brno where he taught, until 1986, at the School of Applied Arts.

The Hunt Kastner exhibition displays collections of designs that Chatrny published privately in small pamphlet editions, as well as photographs, some his own, some by friends and students, that he delightfully ‘enhanced’ with drawn lines and strings. There is great beauty in these small scale works.

chatrny.png

In recognition of the profound influence of Dalbor Chatrny on artists in the Czech Republic (and no doubt in Slovakia before the separation), guests at Friday’s exhibition opening were invited to bring their own small-scale works inspired by the artist’s work, together with some explanatory text. Sadly, few complied, and the few works that were contributed by aspiring or established artists (or perhaps complete unknowns) served only to illustrate how difficult it is to create works of simplicity and beauty that don’t need justification through pretentious conceptualist explanation.

Fine works by Dalibor Chatrny

 

Rubbish (in my opinion) by guests at Friday’s Exhibition Opening

No Dalek

no dalek

Artist David Davros writes: For a child, all is the ‘other’. It is only through a process of secondary familial and public socialisation that a child can construct his or her necessary public and moral structures containing other: LOVE and other: HATE and at the same time  (re)configure his or her symbolic understanding of other: TERRESTRIAL and other: ALIEN. The child will all too often and all too easily reach for a palette of anatomically incomplete, sometimes conventional, sometimes degenerative, quasi-distorting representative forms to defuse the power of the hidden multiverse.

Breaking Wave (oil and modelling clay)

breaking wave

I am fascinated, writes Qi Qi-Di, by the relationship between object and representative medium. The wave breaks, just as its representation breaks, cracking and flaking dynamically as the clay dries. And as our cortical synapses progressively fail, and our own perceptions and memories break and fade,  object, medium, perception: are conjoined in oblivion. In the contemplation of the unending cycle of our striving and failing to convey, lies our only hope for peace, both in ourselves and in the world.

Out of Box

outofbox

Out of Box, writes Ahmed Oud from ArtOffensiv Kolektiv, challenges the underlying strictures of representation, whether figurative or symbolic. Medium may itself remain untransformed by the assertion of the artistic object. Literally, nothing becomes the object when representation lies outside it. It is a message not through art to the world, but from art to the world.

These eccentric contributions have been incinerated (a condition of their contribution, apparently) so you need have no fear that your pleasure in Dalibor Chatrny’s work will be spoiled by the work of his overly cerebral and practically untalented followers.

 

Announcing systems@work Version 6

Share

Version 5 of systems@work’s time@work, expense@work and forms@work was released at the end of 2015. It contained a number of enhancements, most of them enthusiastically received and put to use.

Version 6 will be released in early July. It seems soon for a new major release, but the radical revision of the browser-based Professional Services Workbench warrants it. Gone are the variously muddy-coloured lists of functional options, to be replaced by dynamic tabs, panels, links, and shortcuts.

Our aim has been fourfold:

  • To comply with current standards and graphical fashions
  • To present the user more clearly with To Do lists for data submission and approval
  • To enable navigation with fewer clicks
  • To make navigation pages responses to the user’s device (responsive design)

The result is a Home Page (for time@work) that is more pleasant to look at and easier to use.

Version 6

And list pages that look more modern:

listpage.png

Version 6 will also contain the following enhancements:

  • A new reporting option that allows you to report on and export static data (such as Employees, Clients, Projects and Tasks). This will be of particular benefit for integration.
  • Credit Notes in the browser-based PSW
  • Enhanced emailing options for Invoicing, including an option to select an Employee as a destination and to allow emailing to multiple email addresses
  • Workflow Control for Timesheets to allow diversion of a timesheet from one authoriser to another

 

Money Talks. The Dog Barks. Motivation and Vocation

A few days ago the British Government published a Review on Antimicrobial Resistance . It described the growing resistance of bacteria to over-prescribed antibiotics and suggested that by 2050 a death from hitherto treatable bacterial infections might occur every three seconds, unless new drugs are discovered. It also urged huge investment in research, including the offering of large prizes (a billion US dollars) to pharmaceutical companies  for every new antibiotic they discover. But these vast prizes are insignificant compared to the cost of medicine being thrown back into the dark ages by recalcitrant bacteria.

It is pointless, as some do, to rail against the rapacious selfishness of pharmaceutical companies and expect them to undertake difficult and commercially risky research without incentives.

The report also urged doctors to prescribe fewer antibiotics, and to avoid prescribing antibiotics for infections that are viral. (Here in the Czech Republic there’s a tendency (admittedly now declining) to prescribe antibiotics as a precaution, to prevent a viral cold from turning into a bacterial cough.)

For many years (perhaps even decades) GPs in the UK have been asked to prescribe fewer antibiotics but are understandably upset when patient satisfaction declines. Nevertheless, data from the NHS were published yesterday that showed that doctors had prescribed 7%fewer antibiotics in the last year than during the preceding year.

How was this achieved? Through financial incentives. Some doctors’ practices were awarded 20,000 GBP bonuses, in effect for doing less.

greedydoctor

Money talks. Or as the Hungarians say – Money Talks, the Dog Barks.

We don’t like to think that financial incentives play an important role in motivating practitioners of vocational professions. The decisions of doctors, nurses, teachers and priests are not, we like to suppose, influenced by personal financial gain.  Decisions must always be made on the merits of each case.

But we know that the priest will do us a better wedding or baptation, the nurse’s hands will be softer and the doctor will see us sooner if we oil the wheels a little.

Here in Eastern Europe from 1948 until 1989 society was inspired by the utopian ideal that each of us would eagerly give ‘according to his ability to each according to his need.’ It didn’t work.

We have to accept that money motivates. If you’re managing a company, you must take account of dozens of factors when deciding how to motivate your staff, but if the money is insufficient, none of the other tricks will work – whether you’re juggling with job title, benefits (such as a company car), training, or responsibility. Even if staff say that the ‘money doesn’t really matter,’ they’ll still agree that money is a kind of recognition, whatever uses it’s put to.

You’ve got to accept that money is an incentive, and when you’re devising your incentives you’ve got to be careful. The NHS in the UK, I believe, will reward doctors for diagnosing and prescribing pills for blood pressure, and reward them for not prescribing antibiotics. The rules of the game have got to be sophisticated enough to achieve the right result but not so complex as to confuse,  or be beyond efficient measurement. And there are always the unintended, unanticipated effects of motivation that solve one problem and create another.

But sometimes incentives go too far. My mother is currently  being hounded by her young GP (probably recently married with a kid to support), who claims she’s diabetic, and, judging by the frequency with which she tells me this, it has disturbed her peace of mind.  She’s used a home-diagnostic kit to prove him wrong. Very likely he’d collect a bonus for adding her to the list of Type-2 diabetics, but he’s not going to get away with it.

Money talks. The dog barks. It will always be so. And if it reduces the risk that antibiotics will fail, who cares?

 

 

 

 

Philosophers for Brexit

I read on the news this morning that Britain’s military establishment (or, rather, Britain’s former military establishment) has come out in favour of Brexit. Dozens of former generals have signed a letter arguing that what matters when it comes to defence of the realm is NATO not the EU.

Historians have come out for In. Actors, artists and other luvvies have come out for In. Economists have come out for both, of course, but what should we expect? It is the fashion for groups of all kinds to hold hands and write to The Times in favour of either In or Out. Where do campanologists stand? Ornithologists? Kleptomaniacs? Nymphomaniacs? Meteorologists? Numismatists? Philatelists? Dog lovers?

philosophers

But another very important group has also nailed its colours to the mast today. Less well reported, but surely of greater import,  is a letter in today’s edition of Mind, the journal of the British philosophical establishment, signed by members of Britain’s philosophical community (note that there is no such thing as a former philosopher, unless you mean a dead one). They have come out, albeit quietly, for Out. Entitled ‘But it doesn’t mean anything’ the letter decries the philosophical assumptions on which the EU is built.

Brian Goodlittle, Reader in Philosophical Energetics at Bradford University says, ‘I was approached by the editor of Mind and was asked to sign this letter. I did so enthusiastically. I am fundamentally opposed to the continental drift of modern European philosophy. It favours meaningless nonsense so it’s not actually philosophy at all. I favour the bracing style of British Empiricism. It admits no blather, no metaphysical indecency. During the Second World War British Empiricism was one of the fiercest weapons in our intellectual arsenal. It had few uses on the front line, admittedly, but it helped us to break the Enigma code and, with the help of the Yanks, to build the Bomb, whilst the Nazis were literally dreaming up nonsense. It would be a disgrace if we gave in now to continental so-called philosophies such as phenomenalism, existentialism, structuralism and other forms of poppycock. French philosophy, in particular, is a load of merde, in my opinion. It reeks of garlic and doesn’t make a single iota of sense. Let’s face it, Mate, what does ‘European Union’ mean anyway?’

Another eminent philosopher, Fiona Fruitington, Professor of Radical Empiricism at Northampton University, has calculated that works of continental philosophy weigh on average four times as much as works by British philosophers. ‘Being and Nothingness,’ she says, ‘I would rather read a DIY manual on shelving. EU law is just the same. Voluminous, meaningless and impractical.’

British philosophy has for centuries been tethered to good old common sense. You can only understand a statement if it can be verified, Alfred Ayer told us (though he could never quite explain how this claim could itself be verified). Austrian born British philosopher Karl Popper turned the same idea on its head and said that something only makes sense if it can be falsified (science proceeds that way, he pointed out, rather than by verification, but he never clarified exactly how his own claim could be falsified).

The greatest of them all, my hero Ludwig Wittgenstein, said we must look at how we come to understand language and the meaning of the terms it contains. We must examine language ‘games’ in real human communities (though I don’t think he had the EU in mind). Continental nonsense, and most of what the EU has to say, he would describe as ‘language gone on holiday’, ordinary words extrapolated way beyond their safe and practical usage. The role of the philosopher, he believed, is to show the fly the way out of the bottle, the fly being the ordinary man or woman befuddled by EU terminology.

‘Ever greater union.’ What does ever actually mean, Wittgenstein might ask. How have we come to agree, as a community of minds, on its deployment? And how could we begin to understand the many meanings of ‘sovereignty’ and ‘subsidiarity’?

I have great sympathy for philosophers, but in the end I’m not with them on this. When it comes down to it the vast majority of them don’t know how to boil an egg.

 

Euphoria

I’ve never felt more glad to be alive than when I survived a parachute jump in 1982. ‘Survived’, perhaps, exaggerates the danger. There was none, we were told, but I was so afraid that I barely slept the night before. To this day, I’m not sure why I, and about four other close friends, did it. What were we trying to prove to each other or ourselves? It might have been because the sister of one of my friends dared us to, or because it was the kind of thing no one would have expected us to do. We were hardly paragons of macho masculinity.

As it happens, that sister of one of my friends was very nearly killed when her fixed line caught beneath her chin as she jumped. So, perhaps, there were dangers after all. The instructor and dispatcher in the plane were immediately fired.

parachute 1

We jumped from only 1,000 metres on fixed lines, so our parachutes opened as soon as we left the aircraft (you can’t be trusted to open your own if it’s your very first time), but it was probably doing it alone that scared me. There were two moments of sheer terror – the moment of stepping, unnaturally, into nothing (this was so terrifying that my mind never registered it at all and it remains a blank), and the moment of landing. The two minutes, in between, of floating gently in summer sunshine towards the green and pleasant Herefordshire countryside, were blissful, though my apprehension grew as the land rose rapidly towards me. The greatest danger for the amateur is landing. A colleague, much older than I was then, who jumped for charity a few weeks later, broke both his legs. I landed without incident.

I have never since felt such euphoria, have never since felt so strongly that I would rather be alive than dead, not even when the dentist gives my teeth a clean bill of health. I understand how people become junkies for danger, and I can only imagine that surviving real danger, when bullets are flying, and bombs falling, is an even bigger kick. War reporters, soldiers, sky divers – I can understand, why they want to do what they do. Even so, I have never wanted to jump into empty space again. Once was enough. Euphoria is probably best remembered, not re-experienced.

My partner and a friend of ours jumped from 4,000 metres on Saturday, at an airfield near Kolin in the Czech Republic, which is what has brought my own experience back to me. They didn’t jump alone. They were attached, separately, to professional parachutists (and, it seems, skilful filmmakers) and needed no training. They fell for a minute or two before their parachutes opened and they floated gently downwards for another five to make a soft sliding landing in the grass. Easy peasy. Would I do it again? NO.

See it here:

Mad Jump

 

 

Subversion and Incompetence – the Chilcot Inquiry

When Tony Blair’s embarrassingly titled memoir A Journey was published some years ago I was one of those who was delighted to find it in the Crime section of my favourite bookshops. The most ardent Stop the War campaigners would remove it from Biography or Politics and put it where it truly belonged.

blair

I was against the Iraq War. I’d read a number of books by weapons inspectors and was convinced, as were most of us, that the weapons of mass destruction claims, and the Al Quaida links claims, were nonsense. It also seemed very likely that whatever might follow would be at least as bad as Saddam Hussein’s regime. I therefore looked forward to the Chilcot Inquiry and was an avid spectator as the great and the good were hauled over the coals (sometimes rather gently) by John Chilcot and his team, which included the great historian Martin Gilbert, who, sadly, died before the Inquiry completed its work.

What was astonishing, though unsurprising, was how eloquently, articulately and unashamedly Tony Blair, and his colleagues, defended the Government and its decisions. I saw Blair perform at the Leveson Inquiry into Press Freedom too, and was equally impressed. The man was utterly convinced of his rightness. What is it that enables Great Leaders to fend off criticism so smoothly?

The answer is Zeal – than which there is nothing more dangerous in the world (putting aside the forces of nature).

The zealot rises above and beyond the evidence to promote his or her brand of faith, whether religious or political, or both. Righteousness and moral certainty, often impervious to evidence and doubt, are as dangerous as any weapons of mass destruction. Zealots live in another world. Indeed, so elevated is Blair’s state of mind that he is unaware that ‘It’s a journey’ is a cliché or that LOL means Laugh Out Loud, not Lots of Love. He has no sense of the ordinary.

There are three great political events this year which I look forward to with huge excitement – the EU Referendum, the US Presidential Election, and the publication, on July 6th of the Chilcot Inquiry Report. I sympathise with those who castigate John Chilcot for his tardiness, but it is much more important that the report be exhaustive, comprehensive, and invulnerable to accusations of sloppiness, than that it should be rushed into print. The report will run to two and a half million words. That’s a modest tome after seven years of work.. It would only take me around  four additional years to blog the same number of words (650 words daily every day of the year).

Like many others, I have feared that the mandarins would preside over a whitewash, but if yesterday’s leaks are anything to go by, the Inquiry will deliver stinging rebukes to Tony Blair, Jack Straw (former Foreign Secretary) and Richard Dearlove (former head of MI6) in particular. The first for subverting Cabinet Government and going to war with buddy George without good cause, the second for Straw’s incompetent management of the aftermath, and the last for Dearlove’s failure to stop Blair’s Government from publishing the highly misleading dodgy dossier, which made the case for war on the basis of unsubstantiated claims that Saddam Hussein possessed weapons of mass destruction and could use them against Britain in 45 minutes. It’s probably Blair’s subversion that has rankled most with the mandarins. The Butler report also took him to task for his presidential style of government.

Let’s hope the leaks are not themselves misleading. If they are not Mr Blair, the worst part of your journey has yet to come.