We primarily utilize Qlik Compose for development purposes, enabling us to supply data to business users. We employ transformation logic for the ETL processes, leveraging the staging layer data extracted from our source systems. Subsequently, we utilize this information, aligned with the business logic, to furnish Power BI reports to the users.
When processing data from certain tables with a large volume of data, we encounter significant delays. For instance, when dealing with around one million records, it typically takes three to four hours. To address this, I aim to implement performance improvements across all tables, ensuring swift processing similar to those that are currently complete within seconds. The performance issue primarily arises when we analyze the inserts and updates from the source, subsequently dropping the table. While new insertions are handled promptly, updates are processed slowly, leading to performance issues. Despite consulting our Qlik vendors, they were unable to pinpoint the exact cause of this occurrence. Consequently, I am seeking ways to optimize performance within Qlik Compose, specifically concerning updates.
