[Leaplist] coding - rdbms, revisions, and fks

[Bryan J. Smith]

Bah, I'm dealing with a half-petabyte of flat file storage, often
small, often (and insanely) numerous (into the hundreds of millions,
and it all boils down to a legacy architecture.  People then do
NFS cross-mounts, multiple finds across GFS filesystems and wonder
why it doesn't scale?

It happens.  What works in the past is hard to kill.  Even the vendor
of the software is having a hard time.  And the people I'm trying to
explain this to are not dumb, they hold PhDs and other credentials,
know methods in and out, etc... (although some due treat me like
a "general IT puke" and don't realize I have dealt with regression,
simulation, statistical methods, etc... programatically in my past
career as either an engineer or direct support engineer in the IT
puke role ;).

People know what they know and change is hard.  And yes, it does
get old when they just try to throw another half-dozen 24-48 core,
192-384GiB memory blades at the problem when the storage
architecture is not only the bottleneck.  But I'm trying to explain to them
that distributed storage has its own set of coherency challenges that
are not the same as local filesystems.

Especially when they forget they are now crunching 50-250x as
much data than they were with that old VMS or Solaris system.

________________________________
From: Phil Barnett <philb at philb.us>

Bah, Bad thinking. This is where things go wrong all the time where I work. Wasting 40% space is obviously no big deal on a 200M table. Then you get into a place like Disney and you wasted 40% of 1T. And then you have to rethink and redesign something that could easily have been right from the beginning.

Bad thinking. Efficiency doesn't come from throwing more hardware at a problem, efficiency comes from the mind.

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.