Using OneStream will, ideally, unify and simplify your data – everything from financial information to planning, reporting, and analytics. Many users, however, find that loading data using multiple queries takes a lot longer than they expected, often hours instead of minutes. This defeats the purpose of using OneStream when using it to improve business decision-making and speed up reporting and analytics.
The problem lies in how data units are interconnected in OneStream. Each one represents a unit of work for loading, clearing, calculating, storing, and locking data. That part is fairly simple and should mean that just one data process is running when you load data.
But that’s not always what happens. When loading multiple data-unit imports into the same entity using the AllChannelInput workflow channel, each time a Load Cube step is executed, a sibling Load Cube step also runs. This happens because the load engine needs to evaluate the data unit and ensure it wasn’t loaded by another process. This, however, slows down data loads in the workflow channel.
For example, an organization needs to load 65 data units from 65 different queries of its enterprise data warehouse to a single entity. This requires each query to be loaded by a base-level import workflow. When one data import is processed, it triggers the reloading of all other imports in its parent workflow.
While the Import and Transform steps for all queries in this example only take about 45 minutes, the Load Cube step can take as long as four hours. This is because each of the Load Cube steps launches 65 Load Cube processes. That’s 4,225 processes that need to run. Unfortunately, this is normal behavior. It’s not optimal, but it’s the reality.
If you change the way you leverage Workflow Channel, you can accelerate multiple data loads, which can greatly optimize system performance.
Reducing Data Loads with Workflow Channel
If you want to reduce the load time of data units, you will need to designate each query as a distinct and separate data unit. This can be done using the Workflow Channel feature, and it will treat each query as an independent data unit. Therefore, the Load Cube for that query won’t trigger other Load Cube processes, which can take hours to complete.
To do this, first create a new Workflow Channel in OneStream from the Application tab:
Then assign the new Workflow Channel to all the Import Workflows:
In the Application Properties tab, assign UD1 to the UD Dimension Type for Workflow Channels:
Next, create a dedicated member in the UD1 Data Source dimension for each query:
Finally, assign each member from the Account dimension to the new Workflow Channel created in the first step:
Make sure that each query sends data to its dedicated UD1 member. In addition, create a parent for the new, dedicated UD1 members to make it easier for users to find data and assign all accounts that receive data from the queries to the new Workflow Channel.
It may seem like you’re adding an extra step when using Workflow Channel, but it actually will speed up data loads. This, in turn, will allow you to reap the full benefits of OneStream: easy access to data, automated financial processes, more accurate reports, and a single interface for financial and operational data without the hours of processing time.
Find more information on how to leverage OneStream.