Posts Tagged: ‘puredata’

How to get the most out of your PureData System for Analytics using Hadoop as a cost-efficient extension

23. Juni 2015 Posted by Ralf Götz

Today’s requirements for collecting huge amounts of data are different from several years back when only relational databases satisfied the need for a system of record. Now, new data formats need to be acquired, stored and processed in a convenient and flexible way. Customers need to integrate different systems and platforms to unify data access and acquisition without losing control and security.

The logical data warehouse

More and more relational databases and Hadoop platforms are building the core of a Logical Data Warehouse in which each system handles the workload which it can handle best. We call this using “fit for purpose” stores.

An analytical data warehouse appliance such as PureData System for Analytics is often at the core of this Logical Data Warehouse and it is efficient in many ways. It can host and process several terabytes of valuable, high-quality data enabling lightning fast analytics at scale. And it has been possible (with some effort) to move bulk data between Hadoop and relational databases using Sqoop – an open source component of Hadoop. But there was no way to query both systems using SQL – a huge disadvantage.

Two options for combining relational database and Hadoop

Why move bulk data between different systems or run cross-systems analytical queries? Well, there are several use cases for this scenario but I will only highlight two of them based on a typical business scenario in analytics.

The task: an analyst needs to find out how the stock level of the company’s products will develop throughout the year. This stock level is being updated very frequently and produces lots of data in the current data warehouse system implemented on PureData System for Analytics. Therefore the data cannot be kept in the system for more than a year (hot data). A report on this hot data indicates that the stock level is much too high and needs to be adjusted to keep stock costs low. This would normally trigger immediate sales activities (e.g. a marketing and/or sales campaign with lower prices).

“We need a report, which could analyze all stock levels for all products for the last 10+ years!”

Yet, a historical report, which could analyze all stock levels for all products for the last 10+ years would have indicated that the stock level at this time of the year is a good thing, because a high season is approaching. Therefore, the company would be able to sell most of their products and satisfy the market trend. But how can the company provide such a report with so much data?

Bild

The company would have 2 use case options to satisfy their needs:

  1. Replace the existing analytical data warehouse appliance with a newer and bigger one (This would cost some dollars and has been covered in another blog post.), or
  2. Use an existing Hadoop cluster as a cheap storage and processing extension for the data warehouse appliance (Note that a new, yet to be implemented Hadoop cluster would probably cost more than a bigger PureData box as measured by Total Cost of Ownership).

Option 2 would require a mature, flexible integration interface between Hadoop and PureData. Sqoop would not be able to handle this, because it requires more capabilities than just bulk data movement capabilities from Hadoop to PureData.

IBM Fluid Query for seamless cross-platform data access using standard SQL

These requirements are only two of the reasons why IBM has introduced IBM Fluid Query in March, 2015 as a no charge extension for PureData System for Analytics. Fluid Query enables bulk data movement from Hadoop to PureData and vice versa ANDoperational SQL query federation. With Fluid Query, data residing in Hadoop distributions from Cloudera, Hortonworks and IBM BigInsights for Apache Hadoop can be combined with the data residing in PureData using standard SQL syntax.

“Move and query all data, find the value in the data and integrate only if needed.”

This enables users to seamlessly query older, cooler data and hot data without the complexity of data integration with a more exploratory approach: move and query all data, find the value in the data and integrate only if needed.

Bild

IBM Fluid Query can be downloaded and installed as a free add-on for PureData System for Analytics.

Try it out today. IBM Fluid Query is technology that is available for PureData System for Analytics.  Clients can download and install this software and get started right away with these new capabilities.  Download it here on Fix Central. Doug Dailey’s “Getting Started with Fluid Query” blog for more information and documentation links to get started is highly recommended reading.

Bild

Do you need more information? Follow me on Twitter.

How to get the most out of your PureData System for Analytics using Hadoop as a cost-efficient extension

23. Juni 2015 Posted by Ralf Götz

Today’s requirements for collecting huge amounts of data are different from several years back when only relational databases satisfied the need for a system of record. Now, new data formats need to be acquired, stored and processed in a convenient and flexible way. Customers need to integrate different systems and platforms to unify data access and acquisition without losing control and security.

The logical data warehouse

More and more relational databases and Hadoop platforms are building the core of a Logical Data Warehouse in which each system handles the workload which it can handle best. We call this using “fit for purpose” stores.

An analytical data warehouse appliance such as PureData System for Analytics is often at the core of this Logical Data Warehouse and it is efficient in many ways. It can host and process several terabytes of valuable, high-quality data enabling lightning fast analytics at scale. And it has been possible (with some effort) to move bulk data between Hadoop and relational databases using Sqoop – an open source component of Hadoop. But there was no way to query both systems using SQL – a huge disadvantage.

Two options for combining relational database and Hadoop

Why move bulk data between different systems or run cross-systems analytical queries? Well, there are several use cases for this scenario but I will only highlight two of them based on a typical business scenario in analytics.

The task: an analyst needs to find out how the stock level of the company’s products will develop throughout the year. This stock level is being updated very frequently and produces lots of data in the current data warehouse system implemented on PureData System for Analytics. Therefore the data cannot be kept in the system for more than a year (hot data). A report on this hot data indicates that the stock level is much too high and needs to be adjusted to keep stock costs low. This would normally trigger immediate sales activities (e.g. a marketing and/or sales campaign with lower prices).

“We need a report, which could analyze all stock levels for all products for the last 10+ years!”

Yet, a historical report, which could analyze all stock levels for all products for the last 10+ years would have indicated that the stock level at this time of the year is a good thing, because a high season is approaching. Therefore, the company would be able to sell most of their products and satisfy the market trend. But how can the company provide such a report with so much data?

Bild

The company would have 2 use case options to satisfy their needs:

  1. Replace the existing analytical data warehouse appliance with a newer and bigger one (This would cost some dollars and has been covered in another blog post.), or
  2. Use an existing Hadoop cluster as a cheap storage and processing extension for the data warehouse appliance (Note that a new, yet to be implemented Hadoop cluster would probably cost more than a bigger PureData box as measured by Total Cost of Ownership).

Option 2 would require a mature, flexible integration interface between Hadoop and PureData. Sqoop would not be able to handle this, because it requires more capabilities than just bulk data movement capabilities from Hadoop to PureData.

IBM Fluid Query for seamless cross-platform data access using standard SQL

These requirements are only two of the reasons why IBM has introduced IBM Fluid Query in March, 2015 as a no charge extension for PureData System for Analytics. Fluid Query enables bulk data movement from Hadoop to PureData and vice versa ANDoperational SQL query federation. With Fluid Query, data residing in Hadoop distributions from Cloudera, Hortonworks and IBM BigInsights for Apache Hadoop can be combined with the data residing in PureData using standard SQL syntax.

“Move and query all data, find the value in the data and integrate only if needed.”

This enables users to seamlessly query older, cooler data and hot data without the complexity of data integration with a more exploratory approach: move and query all data, find the value in the data and integrate only if needed.

Bild

IBM Fluid Query can be downloaded and installed as a free add-on for PureData System for Analytics.

Try it out today. IBM Fluid Query is technology that is available for PureData System for Analytics.  Clients can download and install this software and get started right away with these new capabilities.  Download it here on Fix Central. Doug Dailey’s “Getting Started with Fluid Query” blog for more information and documentation links to get started is highly recommended reading.

Bild

Do you need more information? Follow me on Twitter.

How to get the most out of your PureData System for Analytics using Hadoop as a cost-efficient extension

23. Juni 2015 Posted by Ralf Götz

Today’s requirements for collecting huge amounts of data are different from several years back when only relational databases satisfied the need for a system of record. Now, new data formats need to be acquired, stored and processed in a convenient and flexible way. Customers need to integrate different systems and platforms to unify data access and acquisition without losing control and security.

The logical data warehouse

More and more relational databases and Hadoop platforms are building the core of a Logical Data Warehouse in which each system handles the workload which it can handle best. We call this using “fit for purpose” stores.

An analytical data warehouse appliance such as PureData System for Analytics is often at the core of this Logical Data Warehouse and it is efficient in many ways. It can host and process several terabytes of valuable, high-quality data enabling lightning fast analytics at scale. And it has been possible (with some effort) to move bulk data between Hadoop and relational databases using Sqoop – an open source component of Hadoop. But there was no way to query both systems using SQL – a huge disadvantage.

Two options for combining relational database and Hadoop

Why move bulk data between different systems or run cross-systems analytical queries? Well, there are several use cases for this scenario but I will only highlight two of them based on a typical business scenario in analytics.

The task: an analyst needs to find out how the stock level of the company’s products will develop throughout the year. This stock level is being updated very frequently and produces lots of data in the current data warehouse system implemented on PureData System for Analytics. Therefore the data cannot be kept in the system for more than a year (hot data). A report on this hot data indicates that the stock level is much too high and needs to be adjusted to keep stock costs low. This would normally trigger immediate sales activities (e.g. a marketing and/or sales campaign with lower prices).

“We need a report, which could analyze all stock levels for all products for the last 10+ years!”

Yet, a historical report, which could analyze all stock levels for all products for the last 10+ years would have indicated that the stock level at this time of the year is a good thing, because a high season is approaching. Therefore, the company would be able to sell most of their products and satisfy the market trend. But how can the company provide such a report with so much data?

Bild

The company would have 2 use case options to satisfy their needs:

  1. Replace the existing analytical data warehouse appliance with a newer and bigger one (This would cost some dollars and has been covered in another blog post.), or
  2. Use an existing Hadoop cluster as a cheap storage and processing extension for the data warehouse appliance (Note that a new, yet to be implemented Hadoop cluster would probably cost more than a bigger PureData box as measured by Total Cost of Ownership).

Option 2 would require a mature, flexible integration interface between Hadoop and PureData. Sqoop would not be able to handle this, because it requires more capabilities than just bulk data movement capabilities from Hadoop to PureData.

IBM Fluid Query for seamless cross-platform data access using standard SQL

These requirements are only two of the reasons why IBM has introduced IBM Fluid Query in March, 2015 as a no charge extension for PureData System for Analytics. Fluid Query enables bulk data movement from Hadoop to PureData and vice versa ANDoperational SQL query federation. With Fluid Query, data residing in Hadoop distributions from Cloudera, Hortonworks and IBM BigInsights for Apache Hadoop can be combined with the data residing in PureData using standard SQL syntax.

“Move and query all data, find the value in the data and integrate only if needed.”

This enables users to seamlessly query older, cooler data and hot data without the complexity of data integration with a more exploratory approach: move and query all data, find the value in the data and integrate only if needed.

Bild

IBM Fluid Query can be downloaded and installed as a free add-on for PureData System for Analytics.

Try it out today. IBM Fluid Query is technology that is available for PureData System for Analytics.  Clients can download and install this software and get started right away with these new capabilities.  Download it here on Fix Central. Doug Dailey’s “Getting Started with Fluid Query” blog for more information and documentation links to get started is highly recommended reading.

Bild

Do you need more information? Follow me on Twitter.

Speed up SAP Netweaver Business Intelligence queries using IBM PureData System for Analytics (Part 2)

23. Juni 2015 Posted by Ralf Götz

Imagine that you’re the CIO of a big retailer with both an online and store presence. Christmas season is coming, and last year around this time, your critical business processes almost broke because of the heavy demand for deep analytics and frequent reporting during the high season. What would you do?

Bild

In the first post of this series, I described an alternative reporting and analytics solution to SAP Netweaver Business Intelligence, based on IBM PureData System for Analytics, powered by Netezza technology. In order to provide business users with the analytical performance they need to keep up with the growing demand for more data, faster reports, deeper analytics and less cost.

The challenge

A large retail client in Germany had exactly the problem I introduced in the beginning of this post. Their SAP Netweaver Business Intelligence system is essential for certain business critical processes such as month end closure, inventory and demand planning. Under normal conditions, the reporting and analytical workload can be processed in parallel to the more transaction-oriented processes.

But every Monday just before and just after holiday seasons such as Christmas, the system came very close to its maximum capacity—sometimes to capacity overload.

The evaluation process

What would be the best way to tackle the challenge? First, the client needed to evaluate the technical fesibility, the time it would take for a successful implementation and the associated costs.After the evaluation of the different possible approaches, such as upgrading SAP Netweaver Business Intelligence to the latest release, implementing SAP HANA or changing the underlying database out with another, the client decided to implement a sidecar solution to offload the most critical reports and analytics to reduce the workload on SAP Netweaver Business Intelligence.

The solution

The idea was to introduce a purpose built, easy-to-use, high-speed analytics data warehouse appliance that grows with the business requirements: PureData System for Analytics.

Implementation of such a sidecar solution is a best practice approach also recommended by SAP itself with their Sybase IQ columnar database.

After a one week assessment, the top three SAP Netweaver Business Intelligence reports had been identified and run by thousands of users several times every day. Removing this purely analytical workload from the system would guarantee a smooth season next Christmas.

The data (several terabytes) had been integrated with the help of a business partner using the customer’s incumbent extract, transform and load (ETL) platform, which could be connected to SAP and PureData System for Analytics.

For reporting, the client introduced IBM Cognos along with PureData System for Analytics, gaining the maximum out of the new analytics infrastructure.

The result

The most important fact is that our client survived Christmas season (and Easter as well).

Bild

Their SAP Netweaver Business Intelligence system can still serve its purpose, is running smoothly and has been very stable since then. Only the reporting and analytics run now on the sidecar PureData System for Analytics. The response time for typical queries is mostly under two seconds.

Because of the highly flexible implementation of data model and granularity within PureData System for Analytics, the client was even able to increase the frequency of some reports from monthly to weekly updates, which enabled the business users to do more with less effort in a shorter amount of time.

The retailer started the implementation in April 2013 and finished the project in September 2013, on time and on budget.

What do you think of the implementation? Are you facing similar challenges? Let’s connect and follow me on Twitter.

Speed up SAP Netweaver Business Intelligence queries using IBM PureData System for Analytics (Part 2)

23. Juni 2015 Posted by Ralf Götz

Imagine that you’re the CIO of a big retailer with both an online and store presence. Christmas season is coming, and last year around this time, your critical business processes almost broke because of the heavy demand for deep analytics and frequent reporting during the high season. What would you do?

Bild

In the first post of this series, I described an alternative reporting and analytics solution to SAP Netweaver Business Intelligence, based on IBM PureData System for Analytics, powered by Netezza technology. In order to provide business users with the analytical performance they need to keep up with the growing demand for more data, faster reports, deeper analytics and less cost.

The challenge

A large retail client in Germany had exactly the problem I introduced in the beginning of this post. Their SAP Netweaver Business Intelligence system is essential for certain business critical processes such as month end closure, inventory and demand planning. Under normal conditions, the reporting and analytical workload can be processed in parallel to the more transaction-oriented processes.

But every Monday just before and just after holiday seasons such as Christmas, the system came very close to its maximum capacity—sometimes to capacity overload.

The evaluation process

What would be the best way to tackle the challenge? First, the client needed to evaluate the technical fesibility, the time it would take for a successful implementation and the associated costs.After the evaluation of the different possible approaches, such as upgrading SAP Netweaver Business Intelligence to the latest release, implementing SAP HANA or changing the underlying database out with another, the client decided to implement a sidecar solution to offload the most critical reports and analytics to reduce the workload on SAP Netweaver Business Intelligence.

The solution

The idea was to introduce a purpose built, easy-to-use, high-speed analytics data warehouse appliance that grows with the business requirements: PureData System for Analytics.

Implementation of such a sidecar solution is a best practice approach also recommended by SAP itself with their Sybase IQ columnar database.

After a one week assessment, the top three SAP Netweaver Business Intelligence reports had been identified and run by thousands of users several times every day. Removing this purely analytical workload from the system would guarantee a smooth season next Christmas.

The data (several terabytes) had been integrated with the help of a business partner using the customer’s incumbent extract, transform and load (ETL) platform, which could be connected to SAP and PureData System for Analytics.

For reporting, the client introduced IBM Cognos along with PureData System for Analytics, gaining the maximum out of the new analytics infrastructure.

The result

The most important fact is that our client survived Christmas season (and Easter as well).

Bild

Their SAP Netweaver Business Intelligence system can still serve its purpose, is running smoothly and has been very stable since then. Only the reporting and analytics run now on the sidecar PureData System for Analytics. The response time for typical queries is mostly under two seconds.

Because of the highly flexible implementation of data model and granularity within PureData System for Analytics, the client was even able to increase the frequency of some reports from monthly to weekly updates, which enabled the business users to do more with less effort in a shorter amount of time.

The retailer started the implementation in April 2013 and finished the project in September 2013, on time and on budget.

What do you think of the implementation? Are you facing similar challenges? Let’s connect and follow me on Twitter.

Speed up SAP Netweaver Business Intelligence queries using IBM PureData System for Analytics (Part 2)

23. Juni 2015 Posted by Ralf Götz

Imagine that you’re the CIO of a big retailer with both an online and store presence. Christmas season is coming, and last year around this time, your critical business processes almost broke because of the heavy demand for deep analytics and frequent reporting during the high season. What would you do?

Bild

In the first post of this series, I described an alternative reporting and analytics solution to SAP Netweaver Business Intelligence, based on IBM PureData System for Analytics, powered by Netezza technology. In order to provide business users with the analytical performance they need to keep up with the growing demand for more data, faster reports, deeper analytics and less cost.

The challenge

A large retail client in Germany had exactly the problem I introduced in the beginning of this post. Their SAP Netweaver Business Intelligence system is essential for certain business critical processes such as month end closure, inventory and demand planning. Under normal conditions, the reporting and analytical workload can be processed in parallel to the more transaction-oriented processes.

But every Monday just before and just after holiday seasons such as Christmas, the system came very close to its maximum capacity—sometimes to capacity overload.

The evaluation process

What would be the best way to tackle the challenge? First, the client needed to evaluate the technical fesibility, the time it would take for a successful implementation and the associated costs.After the evaluation of the different possible approaches, such as upgrading SAP Netweaver Business Intelligence to the latest release, implementing SAP HANA or changing the underlying database out with another, the client decided to implement a sidecar solution to offload the most critical reports and analytics to reduce the workload on SAP Netweaver Business Intelligence.

The solution

The idea was to introduce a purpose built, easy-to-use, high-speed analytics data warehouse appliance that grows with the business requirements: PureData System for Analytics.

Implementation of such a sidecar solution is a best practice approach also recommended by SAP itself with their Sybase IQ columnar database.

After a one week assessment, the top three SAP Netweaver Business Intelligence reports had been identified and run by thousands of users several times every day. Removing this purely analytical workload from the system would guarantee a smooth season next Christmas.

The data (several terabytes) had been integrated with the help of a business partner using the customer’s incumbent extract, transform and load (ETL) platform, which could be connected to SAP and PureData System for Analytics.

For reporting, the client introduced IBM Cognos along with PureData System for Analytics, gaining the maximum out of the new analytics infrastructure.

The result

The most important fact is that our client survived Christmas season (and Easter as well).

Bild

Their SAP Netweaver Business Intelligence system can still serve its purpose, is running smoothly and has been very stable since then. Only the reporting and analytics run now on the sidecar PureData System for Analytics. The response time for typical queries is mostly under two seconds.

Because of the highly flexible implementation of data model and granularity within PureData System for Analytics, the client was even able to increase the frequency of some reports from monthly to weekly updates, which enabled the business users to do more with less effort in a shorter amount of time.

The retailer started the implementation in April 2013 and finished the project in September 2013, on time and on budget.

What do you think of the implementation? Are you facing similar challenges? Let’s connect and follow me on Twitter.

Speed up SAP Netweaver Business Intelligence queries using IBM PureData System for Analytics (Part 1)

23. Juni 2015 Posted by Ralf Götz

Have you ever encountered the need to accelerate reporting within SAP Business Warehouse (SAP BW)? Did you find a feasible solution that fits your budget and performance requirements? If not, then you might be interested in how to speed up SAP Business Intelligence queries using IBM PureData System for Analytics, powered by Netezza technology. 

Bild

SAP offers a variety of options to help you to improve the performance of your SAP BW queries. These include SAP HANA (as the underlying database, which is an in-memory solution) or SAP Sybase IQ (which is a columnar database working as a sidecar solution). IBM also offers a SAP BW optimized relational database: IBM DB2 for SAP.

But there are additional ways to approach improving performance.

I believe that a more optimal approach is to widen the scope and chose a solution that will provide business intelligence (BI) service consolidating multiple data sources, of which SAP is just one. This should include an evaluation of BI and extraction, transformation and load (ETL) tools, as well as data warehouse appliances (such as IBM PureData System for Analytics, powered by Netezza technology).

From my personal experience with other clients who have reported on data in SAP enterprise resource planning solutions combined with other data sources, I would recommend a target architecture of a downstream enterprise data warehouse, creating a corporate-wide analytical data service built on PureData System for Analytics technology.

Clients I have worked with have achieved significant benefits by extracting and moving the data into PureData System for Analytics rather than trying to extend the capability using existing solutions. The diagram below outlines the high level architectural approach:

Bild

While I do understand that some clients wish to minimize the impact involved in improving the performance of reports in SAP BW when they use SAP Business Explorer or any other compliant BI tool, I strongly believe that the benefits of this approach far outweigh the disadvantages.

Client experiences with SAP Business Warehouse

From my regular discussions with SAP BW clients, I know that many users experience constraints in the areas of:

  • Performance to build the data (InfoCubes)
  • Performance of queries and analysis
  • Time to develop and meet new reporting and analytical requirements
  • Difficulty in incorporating data from non-SAP sources
  • Accessing the data using non-SAP analytical tools

When clients have implemented SAP Business Warehouse Accelerators as a solution to some of these challenges, they have often needed to reduce the amount of data kept in the InfoCubes held on Business Warehouse Accelerators. This is to allow such data to be loaded in a timely manner, maintain performance and avoid excess licensing costs.

In addition, many SAP users are reviewing the best architectural deployment approach for SAP ERP data going forward. Choosing alternative, open approaches such as PureData System for Analytics can prove to be the optimum solution.

The benefits of IBM PureData System for Analytics

IBM clients have found significant benefit from using PureData System for Analytics as their enterprise data warehouse and foundation for data services, gaining a responsive and easy-to-use open business intelligence environment. PureData System for Analytics users are able to choose the best reporting and analytical tools to meet their requirements, consolidating and analyzing data from all sources, both within and outside of the organization. Additionally they have managed to avoid the complexity of having to manage and maintain the SAP environment, which adds additional infrastructure to an already complex environment. Many clients are also gaining a significant competitive advantage through the advanced analytics capabilities within PureData System for Analytics.

PureData System for Analytics clients have been able to:

  • Significantly improve the performance of business intelligence reporting
  • Dramatically reduce ETL time using the power of the PureData System for Analytics database in doing complex transformations
  • Eliminate the need and complexity of loading non-SAP data into SAP BW
  • Load SAP detailed data into the data warehouse, where it can be used for other purposes and subjects areas
  • Retain and analyze historical transaction and master data changes across multiple years and the lowest level of granularity
  • Deliver new projects much more quickly and with less risk due to the simplicity inherent in PureData System for Analytics operations
  • Drastically reduce SAP BW size, mostly eliminating additional hardware investments and license costs

In one of my next blog posts, I will dive into the details of such a project we just put into production at a large German retailer. Comment below if you’d like to share your experiences, or follow me on Twitter.

Speed up SAP Netweaver Business Intelligence queries using IBM PureData System for Analytics (Part 1)

23. Juni 2015 Posted by Ralf Götz

Have you ever encountered the need to accelerate reporting within SAP Business Warehouse (SAP BW)? Did you find a feasible solution that fits your budget and performance requirements? If not, then you might be interested in how to speed up SAP Business Intelligence queries using IBM PureData System for Analytics, powered by Netezza technology. 

Bild

SAP offers a variety of options to help you to improve the performance of your SAP BW queries. These include SAP HANA (as the underlying database, which is an in-memory solution) or SAP Sybase IQ (which is a columnar database working as a sidecar solution). IBM also offers a SAP BW optimized relational database: IBM DB2 for SAP.

But there are additional ways to approach improving performance.

I believe that a more optimal approach is to widen the scope and chose a solution that will provide business intelligence (BI) service consolidating multiple data sources, of which SAP is just one. This should include an evaluation of BI and extraction, transformation and load (ETL) tools, as well as data warehouse appliances (such as IBM PureData System for Analytics, powered by Netezza technology).

From my personal experience with other clients who have reported on data in SAP enterprise resource planning solutions combined with other data sources, I would recommend a target architecture of a downstream enterprise data warehouse, creating a corporate-wide analytical data service built on PureData System for Analytics technology.

Clients I have worked with have achieved significant benefits by extracting and moving the data into PureData System for Analytics rather than trying to extend the capability using existing solutions. The diagram below outlines the high level architectural approach:

Bild

While I do understand that some clients wish to minimize the impact involved in improving the performance of reports in SAP BW when they use SAP Business Explorer or any other compliant BI tool, I strongly believe that the benefits of this approach far outweigh the disadvantages.

Client experiences with SAP Business Warehouse

From my regular discussions with SAP BW clients, I know that many users experience constraints in the areas of:

  • Performance to build the data (InfoCubes)
  • Performance of queries and analysis
  • Time to develop and meet new reporting and analytical requirements
  • Difficulty in incorporating data from non-SAP sources
  • Accessing the data using non-SAP analytical tools

When clients have implemented SAP Business Warehouse Accelerators as a solution to some of these challenges, they have often needed to reduce the amount of data kept in the InfoCubes held on Business Warehouse Accelerators. This is to allow such data to be loaded in a timely manner, maintain performance and avoid excess licensing costs.

In addition, many SAP users are reviewing the best architectural deployment approach for SAP ERP data going forward. Choosing alternative, open approaches such as PureData System for Analytics can prove to be the optimum solution.

The benefits of IBM PureData System for Analytics

IBM clients have found significant benefit from using PureData System for Analytics as their enterprise data warehouse and foundation for data services, gaining a responsive and easy-to-use open business intelligence environment. PureData System for Analytics users are able to choose the best reporting and analytical tools to meet their requirements, consolidating and analyzing data from all sources, both within and outside of the organization. Additionally they have managed to avoid the complexity of having to manage and maintain the SAP environment, which adds additional infrastructure to an already complex environment. Many clients are also gaining a significant competitive advantage through the advanced analytics capabilities within PureData System for Analytics.

PureData System for Analytics clients have been able to:

  • Significantly improve the performance of business intelligence reporting
  • Dramatically reduce ETL time using the power of the PureData System for Analytics database in doing complex transformations
  • Eliminate the need and complexity of loading non-SAP data into SAP BW
  • Load SAP detailed data into the data warehouse, where it can be used for other purposes and subjects areas
  • Retain and analyze historical transaction and master data changes across multiple years and the lowest level of granularity
  • Deliver new projects much more quickly and with less risk due to the simplicity inherent in PureData System for Analytics operations
  • Drastically reduce SAP BW size, mostly eliminating additional hardware investments and license costs

In one of my next blog posts, I will dive into the details of such a project we just put into production at a large German retailer. Comment below if you’d like to share your experiences, or follow me on Twitter.

Speed up SAP Netweaver Business Intelligence queries using IBM PureData System for Analytics (Part 1)

23. Juni 2015 Posted by Ralf Götz

Have you ever encountered the need to accelerate reporting within SAP Business Warehouse (SAP BW)? Did you find a feasible solution that fits your budget and performance requirements? If not, then you might be interested in how to speed up SAP Business Intelligence queries using IBM PureData System for Analytics, powered by Netezza technology. 

Bild

SAP offers a variety of options to help you to improve the performance of your SAP BW queries. These include SAP HANA (as the underlying database, which is an in-memory solution) or SAP Sybase IQ (which is a columnar database working as a sidecar solution). IBM also offers a SAP BW optimized relational database: IBM DB2 for SAP.

But there are additional ways to approach improving performance.

I believe that a more optimal approach is to widen the scope and chose a solution that will provide business intelligence (BI) service consolidating multiple data sources, of which SAP is just one. This should include an evaluation of BI and extraction, transformation and load (ETL) tools, as well as data warehouse appliances (such as IBM PureData System for Analytics, powered by Netezza technology).

From my personal experience with other clients who have reported on data in SAP enterprise resource planning solutions combined with other data sources, I would recommend a target architecture of a downstream enterprise data warehouse, creating a corporate-wide analytical data service built on PureData System for Analytics technology.

Clients I have worked with have achieved significant benefits by extracting and moving the data into PureData System for Analytics rather than trying to extend the capability using existing solutions. The diagram below outlines the high level architectural approach:

Bild

While I do understand that some clients wish to minimize the impact involved in improving the performance of reports in SAP BW when they use SAP Business Explorer or any other compliant BI tool, I strongly believe that the benefits of this approach far outweigh the disadvantages.

Client experiences with SAP Business Warehouse

From my regular discussions with SAP BW clients, I know that many users experience constraints in the areas of:

  • Performance to build the data (InfoCubes)
  • Performance of queries and analysis
  • Time to develop and meet new reporting and analytical requirements
  • Difficulty in incorporating data from non-SAP sources
  • Accessing the data using non-SAP analytical tools

When clients have implemented SAP Business Warehouse Accelerators as a solution to some of these challenges, they have often needed to reduce the amount of data kept in the InfoCubes held on Business Warehouse Accelerators. This is to allow such data to be loaded in a timely manner, maintain performance and avoid excess licensing costs.

In addition, many SAP users are reviewing the best architectural deployment approach for SAP ERP data going forward. Choosing alternative, open approaches such as PureData System for Analytics can prove to be the optimum solution.

The benefits of IBM PureData System for Analytics

IBM clients have found significant benefit from using PureData System for Analytics as their enterprise data warehouse and foundation for data services, gaining a responsive and easy-to-use open business intelligence environment. PureData System for Analytics users are able to choose the best reporting and analytical tools to meet their requirements, consolidating and analyzing data from all sources, both within and outside of the organization. Additionally they have managed to avoid the complexity of having to manage and maintain the SAP environment, which adds additional infrastructure to an already complex environment. Many clients are also gaining a significant competitive advantage through the advanced analytics capabilities within PureData System for Analytics.

PureData System for Analytics clients have been able to:

  • Significantly improve the performance of business intelligence reporting
  • Dramatically reduce ETL time using the power of the PureData System for Analytics database in doing complex transformations
  • Eliminate the need and complexity of loading non-SAP data into SAP BW
  • Load SAP detailed data into the data warehouse, where it can be used for other purposes and subjects areas
  • Retain and analyze historical transaction and master data changes across multiple years and the lowest level of granularity
  • Deliver new projects much more quickly and with less risk due to the simplicity inherent in PureData System for Analytics operations
  • Drastically reduce SAP BW size, mostly eliminating additional hardware investments and license costs

In one of my next blog posts, I will dive into the details of such a project we just put into production at a large German retailer. Comment below if you’d like to share your experiences, or follow me on Twitter.

Faszinierend, würde Mr. Spock sagen: IBM PureData Systems

9. Oktober 2012 Posted by Thomas Wedel

PureData Systems angekündigt
 
IBM schreibt die Story der  neuen Systeme mit integriertem Expertenwissen weiter fort. Heute erschien ein neues Kapitel: Das in meinem früheren BlueBlog-Beitrag beschriebene IBM PureApplication System, der Tausendsassa für schnelles und dynamisches  Deployment von Application Workloads, bekommt neue Kollegen zur Seite gestellt:
 
Während der im April 2012 angekündigte "Magic Worker" PureApplication System ein universelles Plattformsystem für transaktionsorientierte Anwendungen darstellt, kommen nun unter der Bezeichnung "IBM PureData Systems" zwei Spezialisten hinzu, die sich auf datenorientierte bzw. Analyse-Intensive Workloads fokussieren. "PureData Systems for Transactions" ist dabei eine Datenbankplattform, die ausschließlich auf Höchstleistungen bei der Bereitstellung transaktionsintensiver Datenservices getrimmt wurde und vor allem im Zusammenspiel mit einem PureApplication System eine hochperformante und extrem adaptionsfähige Infrastruktur bereitstellt. "PureData Systems for Analytics" ist ein Spezialsystem, wenn es um leistungsfähige Analytics-Aufgabenstellungen im Real-Time-Bereich geht, etwa um auffällige Datenkonstellationen bereits zum Zeitpunkt ihres Auftretens und nicht erst ex-post feststellen und darauf reagieren zu können..
 
Beiden neuen PureData Systems ist das Konzept der "Patterns of Expertise" gemeinsam, das in ähnlicher Weise wie schon beim PureApplication System für eine extrem schnelles und adaptives Deployment der jeweiligen Workloads sorgt - oft im Bereich weniger Minuten im Vergleich zu Implementierungszeiten von Tagen oder Wochen bei bisherigen IT-Infrastrukturkonzepten.
 
Details zu diesen neuen Mitgliedern der IBM PureSystems-Familie findet Ihr unter http://www.ibm.com/ibm/puresystems/de/de/pf_puredata.html

Um im Genre meines vorherigen BlueBlog-Posts zum Thema "Magic Worker" zu bleiben: Mr. Spock, der geniale Analytiker und Beherrscher großer Datenmengen, hätte seine helle Freude an den PureData Systems. Wer weiss, vielleicht hat die USS Enterprise ja welche an Bord?