Integrating Hyperion Regression Testing Into Everyday Life

When it comes to performance testing, one size definitely doesn’t fit all. From baseline testing to integrated health checks, there are many options available to organizations who want to test complex enterprise application suites. Unfortunately, far too many businesses see performance testing as a one-off event that’s done whenever they start working with a new environment–and they might not even understand why it’s being done at all.

One particularly important form of performance testing is regression testing: assessing several selected benchmarks at regular intervals in order to find evidence of performance issues. Regression testing and other performance tests can help you look for trends over time, so that you can identify sudden errors or dips in performance and resolve them. This article will tell you everything you need to know about why and how regression tests are performed for complex business software like Oracle Hyperion.

Goals of Regression Testing

The ultimate goal of regression testing is to reduce the risk that you face as an organization in order to detect problems before they occur for real users in production. Of course, you need to find a “sweet spot” between the risk that you’re willing to face and the budget that you’re willing to spend on minimizing that risk.

Before starting regression testing, ask yourself what your risk tolerance is as an organization. Some businesses are comfortable investing less in regression testing and then working overtime to resolve an issue when it appears; others have a lower risk tolerance and want to identify problems before they even affect users.

When regression testing, you should aim to maximize your coverage while minimizing its impact. The more things that you test in your environment, the greater of an impact that you’ll have. To avoid disrupting your operations, you need to be smart about what you’re doing and how you design your tests.

Regression Test Design

For each test that you design, you should be able to clearly state what you’ll learn from it. For example, you may target different tiers of technology with different tests. Issues with the application server or the database may both reveal themselves to the user in the same way, such as a crash or a failure to start. The more easily that you can target each of these discrete tiers with your tests, the better and faster you can recover from the problem.

You should also have tests for the various products, applications and features, such as OBIEE, Planning, HFM, and Essbase for Oracle Hyperion. Look at the business rules, calculations, forms and reports for each tier. Design your tests to have maximum coverage, and identify some key indicators for each product set that you look at.

By decomposing your tests into various groups broken down by tier, application or feature, it’s easier to run only the ones you need at any given point in time. For example, if you’ve just upgraded Oracle Essbase, you probably only need to run the tests related to Essbase, rather than the entire test suite.

Above all, remember that what works for one business won’t work for the next. Every company has a different conception of what “expected,” “optimal” and “acceptable” conditions are. Understand your system and determine what your trigger points are ahead of time.

Executing Regression Tests

Before you begin executing the regression tests that you’ve designed, consider the following questions:

• What infrastructure environments do you need? Ideally, you should run regression tests in production because that’s the environment that real users are in. Running tests in a QA environment, for example, won’t necessarily detect problems that may occur in production.
• Who will do manual testing, and what techniques will be used? Manual testing is usually much less cost-effective than automated testing but is useful in certain cases.
• What tools are needed for the tests? If you’re using manual testing, you could develop a script or use recorders to document keystrokes and clicks in a browser or user interface.
• Who owns the test suite? This person or group will be responsible for maintaining the test suite and revalidating it whenever there’s a new update or patch.
•  How will you maintain tests and record the results? If you use manual testing, the procedure for each test and the results should be extremely well recorded so that they’re repeatable. The results of regression testing are meaningless unless they’re repeatable so that you can make comparisons between different runs.
• What skills are needed for testers? Are end users your only testers, or do you need people with more knowledge? Can one person test the entire environment (Hyperion Financial Management, Hyperion Planning, Essbase, etc.) or do you need experts for each area?

Remediating and Reporting Regression Tests

If your tests are generating binders full of spreadsheets and data, no one is going to read them regardless of how helpful and informative they are. The ultimate results of regression testing should be clear reports and actionable recommendations.

To some degree, this can be helped in the test design phase. Saying that an application is “slow” isn’t helpful if you don’t know whether all applications are experiencing the same issue. Being smart about designing your tests helps you hone in on the problem and remediate it faster.

Once you detect a problem, you may want to rerun the appropriate tests to verify that there’s really an issue, since anomalies do happen. You should also have a defined process for remediating performance issues: how to enter a ticket, who handles it, and what the timeframe is for it to be completed.

For help and support with regression testing, take a look at our software solution, developed by Hyperion experts at Accelatis, which was recently acquired by Datavail. Our APM platform for performance testing can execute comprehensive, accurate regression testing with very little manual labor. Our platform helps your administrators and consultants quickly determine whether they have met their performance goals, and if not, why and who owns the problem.