Haystack 4.0 and the battle of open vs. closed ecosystems

I’ve been living in a bubble. My bubble has been burst.

This week, I was reading about the new version of Haystack, a data modeling standard I believe is vital for smart buildings and a must-have for analytics projects. Then I stepped into a meeting to discuss an actual project. Two different vendors are installing analytics software on the same building—one for monitoring-based commissioning (MBCx) and one for single-pane-of-glass (SPOG) and supervisory control. Neither vendor is using Haystack or any standard data model.

See ya later, bubble.

To me and my colleague, that was a huge problem and a huge red flag. But neither vendor saw it that way. The owner didn’t see it that way. Our questions got brushed to the side.  Before I explain why that’s an issue, let’s back up a bit and discuss what Haystack is. Let’s take an example piece of building system data, “AI_3”, on the Haystack journey…

Before Haystack

Before Project Haystack created the standard, there were many sad little orphan data points without so much as a real name, just something like “AI_3”, and a current value, perhaps “58″.

We can probably guess AI_3 is a sensor if we know “AI” is short for “analog input”. But otherwise, we’re in the dark. And so is the software application trying to use the data. That’s a big problem, and it’s the reason Project Haystack was formed.

After Haystack

Haystack helps us give meaning to AI_3 with the simple approach of applying “tags”. Tags are like #hashtags in the social media world. Let’s say we add tags such as these:

discharge, air, temp, sensor, point, unit:”°F”, equipRef: AHU-2, navName: DAT 

Now our little orphan is no orphan at all. It belongs to the AHU-2 family (#blessed) and represents a discharge air temperature expressed in degrees Fahrenheit produced by a physical sensor.

With that context, we can now do useful stuff with AI_3, such as:

  • compare its current value to the expected current value or setpoint
  • determine whether AHU-2 is on or off
  • assess AHU-2’s control sequences and whether it needs to be re-programmed
  • determine whether AHU-2’s cooling control loop is stable or hunting
  • compare AI_3, or AHU-2 DAT rather, to all the other discharge air temperature sensors in the building or in the portfolio
  • Determine whether AHU-2 is heated by hot water or steam or natural gas
  • Determine which natural gas meter serves AHU-2
  • Determine which VAV downstream of AHU-2 serves the CEO’s office

That’s not all. Because Haystack is an open standard, any system using these tags can talk to any other system that knows the standard. It’s plug-and-play from here on out. Now AI_3 is usable.

Enter Haystack 4

I’ve been implementing Haystack for years now. I’ll be the first to point out the limitations I see in it. Others, such as the creators of the offshoot Brick Schema have done so extensively. However, at first glance, the newest version, Haystack Version 4.0, seems to be great progress towards erasing them. Let’s unpack it.

Haystack 4 adds sophistication to the model. But the key is that it also adds structure. Brian Frank, one of the founders, explains the need for structure well:

Prior to Haystack 4, there was a flat list of terms that represented tags. These terms were used the same way as a hashtag you see on any social media, such as #iot or #data, as a way to quickly find content related to those topics. The challenge, which you probably have noticed, is that the number of tags seems to continue to grow as subtopics of a main topic become popular. Now you see a message with like 30 hashtags at the bottom to try and guarantee that more people will find the message. As more hashtags get added, less relevant content comes up in searches.

It provides this sophistication and structure through a defined taxonomy and ontology.

Taxonomy: a simple hierarchical arrangement of entities where you have a parent-child, or supertype-subtype, relationship. Example: water is a subtype of liquid. Everything in a building can be organized into a tree-like, hierarchical taxonomy.

Ontology: a domain-specific way of defining the relationships between things in a building, including the restrictions between those relationships. Example: an AHU feeds a VAV and a VAV does not feed an AHU.

Now, we’re modeling the entire systems, including all of their properties. Now, a “machine” can begin to understand the building in the same way we do, and start reducing the effort of using the data even further.

Returning to AI_3, a Haystack 4 model allows a machine to infer everything about this point and where it sits in a system of systems. And finally, when the machine understands how the system is put together, it can start to make inferences and learn. Oh, the places we’ll go.

Analytics without Haystack

Since I was in my bubble assuming everyone was adopting Haystack or Brick, I didn’t think I would ever need to write this, but here we go. What would installing analytics look like without open modeling standards? I don’t need to hypothesize, this is exactly what I experienced this week:

  • Accessing data from the software vendor is a pain (the original reason for my meeting).
  • The owner of the building doesn’t own their own data and depends on the vendor, who actually owns the data, to make use of it.
  • The proprietary, siloed-off model that got us into this mess is back. We can’t preach open controls out one side of our mouths and closed analytics and platforms out the other.
  • Each new integration starts from scratch. Building owners will pay for many vendors to do essentially the same thing—make their data useful—but they will all do it in different ways.
  • Integrations will always be labor-intensive, as there’s no agreed-upon way for existing applications to serve the data to new applications without human intervention, and the two “machines” can’t talk to each other without custom drivers to translate between data models.  

In summary, I see two different worlds forming: an open, collaborative ecosystem of applications and the closed, proprietary quest to control and own the data. Which will win?

My take:

  • one vendor will never be able to meet all of a building owner’s smart building, and even analytics, needs
  • until building owners demand fully open ecosystems, vendors will not provide them because it’s better business not to
  • collaborative vendors will innovate faster than siloed vendors, as they can spend more time on developing cool shit and less on integration

What’s your take?


If this article resonated with you, I invite you to subscribe to my weekly(ish) newsletter on smart buildings and analytics. It will keep you up to date on the industry and any new article I write.