Are you thinking through system improvements after the Xmas Terror Attack?

Information Technology in the federal enterprise does not work like it does in Hollywood.   Although there are plenty of success stories to go around, federal IT is more limited and constrained than we would all want, for lots of reasons. Some of the reasons are just do to complexities and limited budgets.  Some of the reasons are for security.  Some of the reasons are because of  the way the government funds its agencies and manages programs. And some of reasons are because we humans have designed things using the wrong models and implemented them to serve workflows that are flawed to begin with.

This last point is something that I hope enterprise technologists all try to improve on for all our systems.

But perhaps the most important systems to consider improving right now are the ones supporting our national security decision-makers in the field.  These systems will very likely be under review in as part of the investigation into the Xmas terror attack.

There are a huge number of data-focused systems involved that could have been part of stopping this event. The most famous of these are the Terrorist Identities Datamart Environment (TIDE) and the Terrorist Screening Data Base (TSDB).  Other data sources that will need to be scrutinized exist at the Department of Homeland Security, Department of State (especially visa applications), and the intelligence community.

We can expect lots of thought will be put into how many very large enterprise grade data systems interconnect and interoperate together as part of the reviews and after actions underway right now.   I’m sure all concerned in the review of what went wrong in the national security community will realize the problems were not IT and the human element is absolutely the most important to understand.  But still there will be more that systems can smartly do to help humans do right.

I have two recommended pieces of reading for anyone involved in studying what went wrong and how to fix it. Both are from the same thinker, Jeff Jonas. Jeff publishes great thoughts on his blog at: http://jeffjonas.typepad.com, and frankly I recommend all enterprise technologists familiarize themselves with all his writings.  But two pieces stand out as being incredibly relevant to systems design to stop back actors:

Perpetual Analytics and You won’t have to ask– data will find data and relevance will find the user.

In the first piece on Perpetual Analytics Jeff describes the approach to data analytics most organizations find themselves in, where data is extracted from existing systems and secondary analysis is conducted on the extracted data across the entire enterprise whenever a question must be answered.  The ocean must be boiled every time a new analysis is conducted.  This boil the ocean approach does not scale.

He uses the term “perpetual analytics” to “…describe the process of performing real-time analytics on data streams.  Think of this like “directing the rain drops” as they fall into the ocean – placing each drop in the right place and measuring the ripples (i.e., finding relationships and relevance to the historical knowledge). Discovery is made during ingestion and relevant insight is published at that magical moment. ”

Check out the post for more, you will be glad you did, I’m sure.

The other piece I’d recommend you study is You won’t have to ask– data will find data and relevance will find the user .   It highlights a very important shift in approaches to getting the right data to find the right data and make relevance that will find a user. Here is how Jeff put it:

“Next generations of information management systems will not principally rely on users dreaming up smart questions to ask computers. Rather, this new breed of technology will make it possible for data to find itself and relevant discoveries to find the consumer (e.g., a user). And all in real time of course. While this will bring with it new policy debates like which data will be permitted to find which data and who is notified of what relevance, I am going to stay focused in this post on what this technology will enable.”

Another thing all of us who have ever worked in the federal IT business know, it can be very hard to change/improve legacy systems and legacy approaches.  But our existing systems are calling out for improvement, and I’m certain Jeff Jonas is spelling out a model that will improve upon the current approach.

Do you agree?

CTOvision Pro Special Technology Assessments

We produce special technology reviews continuously updated for CTOvision Pro members. Categories we cover include:

  • Analytical Tools - With a special focus on technologies that can make dramatic positive improvements for enterprise analysts.
  • Big Data - We cover the technologies that help organizations deal with massive quantities of data.
  • Cloud Computing - We curate information on the technologies enabling enterprise use of the cloud.
  • Communications - Advances in communications are revolutionizing how data gets moved.
  • GreenIT - A great and virtuous reason to modernize!
  • Infrastructure  - Modernizing Infrastructure can have dramatic benefits on functionality while reducing operating costs.
  • Mobile - This revolution is empowering the workforce in ways few of us ever dreamed of.
  • Security  -  There are real needs for enhancements to security systems.
  • Visualization  - Connecting computers with humans.
  • Hot Technologies - Firms we believe warrant special attention.

 

Recent Research

Pew Report: Increasing Technology Use among Seniors

Finding The Elusive Data Scientist In The Federal Space

DoD Public And Private Cloud Mandates: And insights from a deployed communications professional on why it matters

Intel CEO Brian Krzanich and Cloudera CSO Mike Olson on Intel and Cloudera’s Technology Collaboration

Watch For More Product Feature Enhancements for Actifio Following $100M Funding Round

Navy Information Dominance Corps: IT still searching for the right governance model

DISA Provides A milCloud Overview: Looks like progress, but watch for two big risks

Innovators, Integrators and Tech Vendors: Here is what the government hopes they will buy from you in 2015

Navy continues to invest in innovation: Review their S&T efforts here

MSPA Unified Certification Standard For Cloud Service Providers: Is This A Commercial Version of FedRamp?

Watch Ben Fry And His Visualizations: Multiple use-cases come to mind, including national security efforts

Agenda And More Details for 4-5 March NIST Data Science Symposium

solid
About Bob Gourley

Bob Gourley is the publisher of CTOvision.com and DelphiBrief.com and the new analysis focused Analyst One Bob's background is as an all source intelligence analyst and an enterprise CTO. Find him on Twitter at @BobGourley

  • http://greylogic.us/ Jeffrey Carr

    Hi Bob,Jeff Jonas is one of the shining beacons of innovation living today in my view. Thanks for linking to his posts. For what its worth, my contribution (which I've named Perpetual Threat Analysis) to solving the airport screening process can be read here: http://bit.ly/5kVwu7

  • http://jbordeaux.com/ John Bordeaux

    Not only do I agree, but other countries are doing this – and segments within the IC are doing this. Intellipedia, with the outstanding upgrades suggested in Living Intellligence (http://www.youtube.com/watch?v=nbgQ1V2BLEs), could be the start of an organic approach to 'databases.' Singapore (read Thomas Quinlan's "Seeing the Invisible") has a holistic approach devoted to sensing and propagating 'weak signals' in the information environment (Rapid Assessment Horizon Scanning) instead of some outmoded counter-terrorism 'database.' Old notions of databases, centralized 'integration centers,' and single portals have been eclipsed by the problem and visionaries who get it. Instead, the U.S. stumbles on: sticking to 30 year-old-notions of "getting the right info to the right person" nonsense – as if a Command Center knew what was relevant for every stakeholder; believing a single "integration" activity sitting outside the market of intelligence agencies will 'bring it all together;' procuring Agency-specific IT and then trusting in the kindness of strangers to somehow 'bring it all together,' etc. Oh, and let's not forget our friends in Congress: de-funding the only program dedicated to redesigning the national security system, the Project on National Security Reform. Over two years in existence, with a Who's Who advisory council, who have issued solid recommendations about addressing the real reasons for system failure. Zero'd out in this year's budget.We won't be lucky forever. I wonder if we'll learn in time. There are voices and heroes who knew before Christmas that the system was in failure. When will someone at the Deputies level have the courage to listen? Or will we always be more comfortable pointing the finger at a 'sister' agency? Who sleeps well at night knowing they successfully passed the blame to a fellow American?

  • http://ctovision.com ctovision

    Jeffrey thanks for the comment and the link. I enjoyed reading your post and I think you bring out some great points about the real way to deal with this, by applying smart humans to the problem.

  • http://jbordeaux.com/ John Bordeaux

    Not only do I agree, but other countries are doing this – and segments within the IC are doing this. Intellipedia, with the outstanding upgrades suggested in Living Intellligence (http://www.youtube.com/watch?v=nbgQ1V2BLEs), could be the start of an organic approach to 'databases.' Singapore (read Thomas Quinlan's "Seeing the Invisible") has a holistic approach devoted to sensing and propagating 'weak signals' in the information environment (Rapid Assessment Horizon Scanning) instead of some outmoded counter-terrorism 'database.' Old notions of databases, centralized 'integration centers,' and single portals have been eclipsed by the problem and visionaries who get it. Instead, the U.S. stumbles on: sticking to 30 year-old-notions of "getting the right info to the right person" nonsense – as if a Command Center knew what was relevant for every stakeholder; believing a single "integration" activity sitting outside the market of intelligence agencies will 'bring it all together;' procuring Agency-specific IT and then trusting in the kindness of strangers to somehow 'bring it all together,' etc. Oh, and let's not forget our friends in Congress: de-funding the only program dedicated to redesigning the national security system, the Project on National Security Reform. Over two years in existence, with a Who's Who advisory council, who have issued solid recommendations about addressing the real reasons for system failure. Zero'd out in this year's budget.We won't be lucky forever. I wonder if we'll learn in time. There are voices and heroes who knew before Christmas that the system was in failure. When will someone at the Deputies level have the courage to listen? Or will we always be more comfortable pointing the finger at a 'sister' agency? Who sleeps well at night knowing they successfully passed the blame to a fellow American?

  • http://ctovision.com ctovision

    Jeffrey thanks for the comment and the link. I enjoyed reading your post and I think you bring out some great points about the real way to deal with this, by applying smart humans to the problem.

  • http://ctovision.com ctovision

    John thanks much for the comments and the link to the Living Intelligence video. I also agree, of course, that Congress needs to look at itself. I hear the number of committees with DHS oversight is 88? No one wants to give up any power so there is no motivation to make things better there. That probably contributes to poor funding decision-making.

  • http://ctovision.com ctovision

    John thanks much for the comments and the link to the Living Intelligence video. I also agree, of course, that Congress needs to look at itself. I hear the number of committees with DHS oversight is 88? No one wants to give up any power so there is no motivation to make things better there. That probably contributes to poor funding decision-making.

  • http://greylogic.us/ Jeffrey Carr

    Hi Bob,Jeff Jonas is one of the shining beacons of innovation living today in my view. Thanks for linking to his posts. For what its worth, my contribution (which I’ve named Perpetual Threat Analysis) to solving the airport screening process can be read here: http://bit.ly/5kVwu7

    • http://ctovision.com Bob Gourley

      Jeffrey thanks for the comment and the link. I enjoyed reading your post and I think you bring out some great points about the real way to deal with this, by applying smart humans to the problem.

  • http://greylogic.us/ Jeffrey Carr

    Hi Bob,Jeff Jonas is one of the shining beacons of innovation living today in my view. Thanks for linking to his posts. For what its worth, my contribution (which I’ve named Perpetual Threat Analysis) to solving the airport screening process can be read here: http://bit.ly/5kVwu7

    • http://ctovision.com Bob Gourley

      Jeffrey thanks for the comment and the link. I enjoyed reading your post and I think you bring out some great points about the real way to deal with this, by applying smart humans to the problem.

  • http://greylogic.us/ Jeffrey Carr

    Hi Bob,Jeff Jonas is one of the shining beacons of innovation living today in my view. Thanks for linking to his posts. For what its worth, my contribution (which I've named Perpetual Threat Analysis) to solving the airport screening process can be read here: http://bit.ly/5kVwu7

  • http://jbordeaux.com/ John Bordeaux

    Not only do I agree, but other countries are doing this – and segments within the IC are doing this. Intellipedia, with the outstanding upgrades suggested in Living Intellligence (http://www.youtube.com/watch?v=nbgQ1V2BLEs), could be the start of an organic approach to ‘databases.’ Singapore (read Thomas Quinlan’s “Seeing the Invisible”) has a holistic approach devoted to sensing and propagating ‘weak signals’ in the information environment (Rapid Assessment Horizon Scanning) instead of some outmoded counter-terrorism ‘database.’ Old notions of databases, centralized ‘integration centers,’ and single portals have been eclipsed by the problem and visionaries who get it.

    Instead, the U.S. stumbles on: sticking to 30 year-old-notions of “getting the right info to the right person” nonsense – as if a Command Center knew what was relevant for every stakeholder; believing a single “integration” activity sitting outside the market of intelligence agencies will ‘bring it all together;’ procuring Agency-specific IT and then trusting in the kindness of strangers to somehow ‘bring it all together,’ etc.

    Oh, and let’s not forget our friends in Congress: de-funding the only program dedicated to redesigning the national security system, the Project on National Security Reform. Over two years in existence, with a Who’s Who advisory council, who have issued solid recommendations about addressing the real reasons for system failure. Zero’d out in this year’s budget.

    We won’t be lucky forever. I wonder if we’ll learn in time. There are voices and heroes who knew before Christmas that the system was in failure. When will someone at the Deputies level have the courage to listen? Or will we always be more comfortable pointing the finger at a ‘sister’ agency? Who sleeps well at night knowing they successfully passed the blame to a fellow American?

    • http://ctovision.com Bob Gourley

      John thanks much for the comments and the link to the Living Intelligence video. I also agree, of course, that Congress needs to look at itself. I hear the number of committees with DHS oversight is 88? No one wants to give up any power so there is no motivation to make things better there. That probably contributes to poor funding decision-making.

  • http://jbordeaux.com/ John Bordeaux

    Not only do I agree, but other countries are doing this – and segments within the IC are doing this. Intellipedia, with the outstanding upgrades suggested in Living Intellligence (http://www.youtube.com/watch?v=nbgQ1V2BLEs), could be the start of an organic approach to ‘databases.’ Singapore (read Thomas Quinlan’s “Seeing the Invisible”) has a holistic approach devoted to sensing and propagating ‘weak signals’ in the information environment (Rapid Assessment Horizon Scanning) instead of some outmoded counter-terrorism ‘database.’ Old notions of databases, centralized ‘integration centers,’ and single portals have been eclipsed by the problem and visionaries who get it.

    Instead, the U.S. stumbles on: sticking to 30 year-old-notions of “getting the right info to the right person” nonsense – as if a Command Center knew what was relevant for every stakeholder; believing a single “integration” activity sitting outside the market of intelligence agencies will ‘bring it all together;’ procuring Agency-specific IT and then trusting in the kindness of strangers to somehow ‘bring it all together,’ etc.

    Oh, and let’s not forget our friends in Congress: de-funding the only program dedicated to redesigning the national security system, the Project on National Security Reform. Over two years in existence, with a Who’s Who advisory council, who have issued solid recommendations about addressing the real reasons for system failure. Zero’d out in this year’s budget.

    We won’t be lucky forever. I wonder if we’ll learn in time. There are voices and heroes who knew before Christmas that the system was in failure. When will someone at the Deputies level have the courage to listen? Or will we always be more comfortable pointing the finger at a ‘sister’ agency? Who sleeps well at night knowing they successfully passed the blame to a fellow American?

    • http://ctovision.com Bob Gourley

      John thanks much for the comments and the link to the Living Intelligence video. I also agree, of course, that Congress needs to look at itself. I hear the number of committees with DHS oversight is 88? No one wants to give up any power so there is no motivation to make things better there. That probably contributes to poor funding decision-making.

  • Pingback: DorobekInsider: Welcome 2010 – what you may have missed while we were away « DorobekInsider.com

  • Pingback: CTOvision.com – Enterprise Technology Developments in 2010 and 2011