Using the Right Tools: File Storage, Object Storage, and Intelligent Content Management

File and object storage are those deceptively simple tasks in a software project — it seems straightforward at first (“just store it in the database” or “just use S3”). But then real-world requirements start piling up: permissions, client data segregation, searching, auditing, retention, and more. How much will it cost to scale your solution to production levels of users and documents? How does the system handle tens of thousands or even millions of documents?

Object storage APIs help, but they also bring other challenges. Some APIs are inflexible and limit your ability to integrate your own logic. Other APIs require you to store your data in the cloud, which you may be unwilling (or unable) to do.

yuuvis® Momentum is a document repository that offers extended features for building content and information management solutions, handling all of the file storage concerns of a document management system and more. This modern, cloud-native technology stack supports public cloud, private cloud (including on-premises deployment), and hybrid cloud, enabling you to use the same stack everywhere, flexibly deploy your solution on your own terms, and cost-effectively scale to the most intense workloads.

Let’s look at what makes object storage useful and why choosing yuuvis® Momentum adds additional value over a simple object storage services such as Amazon S3.

Why Use Object Storage?

First, some background: what is object storage? As a simple example, consider a photo saved on your hard disk. Instead of storing a file as a piece of data at a specific location — C:UsersJohnDesktopIMG_0712.JPG — object storage associates each file with a unique ID for retrieval. The system is then free to store that data however and wherever it sees fit.

For example, it might distribute files across multiple disks, allowing the system to grow (more hardware can be added almost seamlessly since the file is accessed by ID, rather than location). Data might be stored in multiple places to enable more fault tolerance (and quicker access across the globe). Data that isn’t accessed for a long time might be transparently moved to cheaper, slower storage.

Object storage techniques are what enable cloud providers to offer services that are so cost-effective and highly scalable. Additionally, that photo described above consists of data and metadata. The data is the actual image itself, but there is other information: when it was taken, the camera model, photo tags, and other similar information about the picture. That is metadata.

Object storage stores and treats files as objects, where the metadata is a first-class component of the object, along with the actual data. Importantly, object storage lets you access the metadata without needing to download the entire file. It can also preserve each different data revision (for example, if the photo is cropped or enhanced).

Object storage really shines when handling a large amount of data that mostly doesn’t change. This category may include more use cases than you think. Imagine a document that needs to be approved and published. Once the document has been uploaded into the system, approving and publishing the document wouldn’t change the contents of the document itself. Whether it is approved and published would likely only change the metadata.

In fact, it may even be a requirement that the document not change. When you start adding requirements such as versioning and retention, this behavior of object storage becomes valuable. Even if the document is revised, you can preserve its different versions over time. If the retention policy requires the document data itself to remain unchanged, you can still capture and update its status, use, or other business-relevant information in its metadata.

Beyond Object Storage, Beyond S3

You might recognize Amazon’s S3 product as a common example of object storage. Why not just use that? S3 is a useful cloud storage offering with a few of the features described earlier. It is very cost effective and easily scales to very large amounts of data. Among other benefits, it allows for files to be replicated across multiple availability zones for speed and reliability, and it has built-in storage tiering (archiving old objects to cheaper storage).

Nevertheless, S3 is still, at its core, just a place to store data. If you want to query and search those files effectively, or apply logic to files, you need to implement that separately. In contrast, yuuvis® Momentum has a sophisticated metadata and type system and tackles far more than just storing data. Imagine you have an invoice, for example. The invoice itself is just a document, but it might require a sign-off from the finance team, and you would want to keep track of whether or not it has been paid. That information could be stored seamlessly alongside the invoice itself (as metadata).

Eliminating the need to build this out yourself, yuuvis® Momentum provides the ingredients to integrate such a system. It could treat the invoice as a first-class object type, along with the tracking information. It even offers workflow integration with Flowable to implement business process management.

Imagine you have not just one invoice, of course, but thousands. The advanced querying facilities in yuuvis® Momentum, for example, enable easy searches for all invoices that haven’t been paid, or all invoices approved by a particular employee. An integrated, full-text extraction and query service also enables you to search through the full content of each document.

An additional feature of yuuvis® Momentum is that all the above can be handled in multi-tenant fashion: in other words, you can ensure all data for each separate customer (or department) is kept separate, while still deploying the same object definitions and logic (schema). At the same time, multiple applications (schemas) can run on the same instance (a single instance can scale up to 10⁹ objects). Schemas can also be tenant-specific.