 |
 |
 |
Functional Testing
Tools |
|
|
| |
|
|
|
|
|
|
| |
| |
View detailed
findings for all Criteria Groups. Please be patient while the data
loads. |
|
|
| |
|
|
|
| |
|
Development |
|
|
|
|
| |
|
What
tools are provided for developing code, content, and web
pages. |
|
| |
|
Factor: Standard |
|
| |
|
Product Name |
Rational Performance
Studio
1.5 |
SilkTest
5.0.2 |
WinRunner
6.0 |
e-Test
Suite
4.2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Manufacturer |
Rational Software |
Segue |
Mercury
Interactive |
RSW Software, Inc. |
|
| |
|
|
Group Score:
142 out of 230
Total Score:
989 out of 1520 |
|
Group Score:
138 out of 230
Total Score:
845 out of 1520 |
|
Group Score:
170 out of 230
Total Score:
816 out of 1520 |
|
Group Score:
136 out of 230
Total Score:
736 out of 1520 |
|
| |
|
Records
Scripts |
|
| |
|
Weight: 3 |
|
Weighted
Score:
16
|
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
| |
|
|
Scripts for both GUI and Vuser are recorded using the
Rational Robot tool. Three types of recording are possible: API,
Proxy, and Network. |
|
All applications are recorded using the SIlkTest application
in the same process. |
|
Scripts are generated with the WinRunner product. To record
web applications, the Web Test add-in must be loaded. The recorder
is simple to use. Scripts can be created and run quickly. |
|
E-Tester is the tool used to record, modify, and run
scripts. |
|
| |
|
Recording
Process |
|
| |
|
Weight: 5 |
|
Weighted
Score:
48
|
|
Weighted
Score:
48
|
|
Weighted
Score:
80
|
|
Weighted
Score:
48
|
|
| |
|
|
Scripts are recorded using Object-Oriented recording which
makes a script more reusable because it does not rely on bitmap
coordinates to locate an object. |
|
SilkTest scripts are created using object-oriented
recording. Scripts recorded using object oriented process are
believed to be more reusable. |
|
WinRunner allows for both Object-Oriented and analog
(bitmap) recording. Giving it a lot more flexibility to test more
types of applications such legacy RTE applications. |
|
Scripts are created using object-oriented
recording. |
|
| |
|
Scripting
Languages Supported |
|
| |
|
Weight: 4 |
|
Weighted
Score:
24
|
|
Weighted
Score:
16
|
|
Weighted
Score:
16
|
|
Weighted
Score:
24
|
|
| |
|
|
SQABasic is a Microsoft Basic-like language used for GUI
scripts. VU is a C-like language that is used for Vuser
scripts. |
|
4Test is an object-oriented language used for SilkTest
scripts. |
|
Test Script Language is a proprietary language provided by
Mercury. |
|
VBScript. Scripts can also be extended using calls to
external functions written in Visual Basic, C, C++, and
Java. |
|
| |
|
Script
Editor |
|
| |
|
Weight: 4 |
|
Weighted
Score:
24
|
|
Weighted
Score:
32
|
|
Weighted
Score:
32
|
|
Weighted
Score:
24
|
|
| |
|
|
An
editor for debugging, testing scripts and adding functionality is
included. |
|
SilkTest provides standard editor features such as debugging
and color coding of keywords. A library browser is available from
the help menu that allows the user to look-up methods associated
with the objects in the script. |
|
The editor has a feature called the Function Generator that
makes it easier to add functions to the script by suggesting a
function or by selecting one from a provided list. Also includes the
standard editor features such as a debugger and
breakpoints. |
|
e-Test Suite Provides an editor to write
scriptlets. |
|
| |
|
Native
APIs Supported |
|
| |
|
Weight: 2 |
|
Weighted
Score:
6
|
|
Weighted
Score:
6
|
|
Weighted
Score:
6
|
|
Weighted
Score:
4
|
|
| |
|
|
Windows APIs, .DLLs or custom-created functions can be
called from the SQABasic language, just as they can in Microsoft
Basic. |
|
Segue's 4Test Language allows for call to native
libraries. |
|
Mercury's Test Script Language (TSL) provides native
libraries to add more functionality to test scripts. |
|
Scriptlets can be added to call native APIs as well as
functions written in VB, C, or Java. |
|
| |
|
User
Defined Functions |
|
| |
|
Weight: 4 |
|
Weighted
Score:
24
|
|
Weighted
Score:
24
|
|
Weighted
Score:
24
|
|
Weighted
Score:
24
|
|
| |
|
|
Both the SQABasic and VU language allow for the creation of
user-defined functions due to the fact that they are so closely
related to Microsoft Basic and the C language. |
|
Custom functions can be created to add
functionality. |
|
TSL allows for the creation of custom functions that can be
called from within the script. |
|
Functions can be written in the form of scriptlets which can
be added to the script to add more specific functionality. |
|
| |
|
|
|
| |
|
Application Types |
|
|
|
|
| |
|
The
types of applications that can be tested (i.e., Web or
Client-Server). |
|
| |
|
Factor: Standard |
|
| |
|
Product Name |
Rational Performance
Studio
1.5 |
SilkTest
5.0.2 |
WinRunner
6.0 |
e-Test
Suite
4.2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Manufacturer |
Rational Software |
Segue |
Mercury
Interactive |
RSW Software, Inc. |
|
| |
|
|
Group Score:
64 out of 80
Total Score:
989 out of 1520 |
|
Group Score:
48 out of 80
Total Score:
845 out of 1520 |
|
Group Score:
80 out of 80
Total Score:
816 out of 1520 |
|
Group Score:
32 out of 80
Total Score:
736 out of 1520 |
|
| |
|
Applications
Supported |
|
| |
|
Weight: 5 |
|
Weighted
Score:
64
|
|
Weighted
Score:
48
|
|
Weighted
Score:
80
|
|
Weighted
Score:
32
|
|
| |
|
|
Windows, Client Server, HTTP, Jolt, Oracle, Socket, ODBC,
Sybase, Tuxedo, SAP, and CORBA |
|
Windows, Client/Server, Web, Database, Java |
|
Windows, Client/Server, Java, Web, RTE, ERP, CRM |
|
Web Only |
|
| |
|
|
|
| |
|
General |
|
|
|
|
| |
|
Basic
information about the product. |
|
| |
|
Factor: Standard |
|
| |
|
Product Name |
Rational Performance
Studio
1.5 |
SilkTest
5.0.2 |
WinRunner
6.0 |
e-Test
Suite
4.2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Manufacturer |
Rational Software |
Segue |
Mercury
Interactive |
RSW Software, Inc. |
|
| |
|
|
Group Score:
150 out of 215
Total Score:
989 out of 1520 |
|
Group Score:
91 out of 215
Total Score:
845 out of 1520 |
|
Group Score:
141 out of 215
Total Score:
816 out of 1520 |
|
Group Score:
111 out of 215
Total Score:
736 out of 1520 |
|
| |
|
Display
OS |
|
| |
|
Weight: 1 |
|
Weighted
Score:
2
|
|
Weighted
Score:
3
|
|
Weighted
Score:
2
|
|
Weighted
Score:
2
|
|
| |
|
|
WIN |
|
WIN - UNIX |
|
WIN |
|
WIN |
|
| |
|
Documentation |
|
| |
|
Weight: 4 |
|
Weighted
Score:
32
|
|
Weighted
Score:
16
|
|
Weighted
Score:
24
|
|
Weighted
Score:
24
|
|
| |
|
|
Documentation is provided in both hard copy and .pdf format.
The numerous manuals provided cover all aspects of the testing
process and explains in great detail the process followed by
Rational. |
|
Documentation includes tutorials for web and windows
applications. The documentation, the analyst believes fails to
demonstrate the full functionality of SilkTest. Training would
definitely be recommended. |
|
Documentation is available in .pdf and hard copy; includes a
useful tutorial that does a good job of pointing out the useful
features of WinRunner. |
|
Each tool comes with its own user manual. Getting started
manual was helpful in learning the process RSW follows to test
software. Available in .pdf format. |
|
| |
|
Ease
of Use |
|
| |
|
Weight: 3 |
|
Weighted
Score:
16
|
|
Weighted
Score:
12
|
|
Weighted
Score:
16
|
|
Weighted
Score:
16
|
|
| |
|
|
When Rational PerformanceStudio is first installed, the
number of tools provided can seem overwhelming. But once the user
begins working with the Rational Robot tool, they should find an
intuitive tool that allows them to accomplish their tasks with
relative ease. The tutorials provided give the user a good
understanding of how the pieces fit together. |
|
SilkTest is very structured in the way in adheres to a
strict testing paradigm. It makes it easy to follow a testing
process of creating test cases that only test on function at a time
and grouping in to a single test plan. |
|
Recording and editing scripts are easily accomplished with
the WinRunner product. The interface is navigated easily. After the
completion of the provided tutorial the user should be able to
record, edit, and run a script in no time. |
|
With e-Test Suite, there is a lot to learn, especially the
script recorder, but the tutorials provided do a good job of
familiarizing the user with the product's functionality and
features. I was able to record a script and test within a few
minutes. |
|
| |
|
Additional
Software Required |
|
| |
|
Weight: 1 |
|
Weighted
Score:
5
|
|
Weighted
Score:
3
|
|
Weighted
Score:
2
|
|
Weighted
Score:
1
|
|
| |
|
|
All needed software is supplied by the Rational
PerformanceStudio. |
|
SilkRadar is needed to provide bug tracking
functionality. |
|
Test Suite is needed for Bug Tracking and Test Plan
integration. |
|
Third party error tracking, data generation and source code
control software if desired. |
|
| |
|
General
Test Paradigm |
|
| |
|
Weight: 1 |
|
Weighted
Score:
3
|
|
Weighted
Score:
3
|
|
Weighted
Score:
3
|
|
Weighted
Score:
4
|
|
| |
|
|
A
repository is created to store the assets for the test. A script is
then recorded using the Robot tool. The script can then be
customized to include custom functions. It is then played back in
the Robot tool. After test completion, results are then viewed in
the Log viewer at which time the failures or bugs can then be
entered into the ClearQuest product. Reports can then be made into
reports and analyzed with the Test Manager product. |
|
A
test plan is created either by hand or by using the provided wizard
which includes adding a test frame with the application to be
tested. A base state is the recorded so SilkTest can reset the
application after each test execution. Scripts are then recorded and
added to the plan where the can be modified to add functionality.
The tests are ran and the results analyzed. |
|
Scripts are created by first running the Rapid Test Script
wizard so the product can learn all the objects associated with the
application or by simply recording a business process and expected
results. Scripts can be edited to add functionality to test certain
features or criteria. Run the test and then analyze the results with
the Results viewer. |
|
The user chooses or creates a workspace to place recorded
scripts. Scripts are then recorded and edited within e-Tester. Very
simple and straight forward. |
|
| |
|
Platform
Tested |
|
| |
|
Weight: 1 |
|
Weighted
Score:
0
|
|
Weighted
Score:
0
|
|
Weighted
Score:
0
|
|
Weighted
Score:
0
|
|
| |
|
|
WIN NT 4.0 SP6 Pentium Pro II 350 MHz 256K Ram |
|
WIN NT 4.0 SP6 Pentium Pro II 350 MHz 256K Ram |
|
WIN NT 4.0 SP6 Pentium Pro II 350 MHz 256K Ram |
|
WIN NT 4.0 SP6 Pentium Pro II 350 MHz 256K Ram |
|
| |
|
Product
Positioning |
|
| |
|
Weight: 1 |
|
Weighted
Score:
0
|
|
Weighted
Score:
0
|
|
Weighted
Score:
0
|
|
Weighted
Score:
0
|
|
| |
|
|
Rational Software provides the industry's only team unifying
solutions, encompassing market-leading tools for all team members,
integrated best practices arising from more than 18 years of
software engineering leadership, and comprehensive professional
services. |
|
SilkTest from Segue Software is the industry's leading
regression testing product for e-business applications. Whether
you're testing a Web, Java or traditional client/server application,
SilkTest supports your organization's needs. |
|
WinRunner is an integrated, functional testing tool for your
entire enterprise. By capturing, verifying, and replaying user
interactions automatically, WinRunner identifies defects and ensures
that business processes, which span across multiple applications and
databases, work flawlessly the first time and remain
reliable. |
|
e-Tester is the only functional test tool that doesn't force
you to become an expert programmer in order to write comprehensive
tests. |
|
| |
|
System
Requirements |
|
| |
|
Weight: 2 |
|
Weighted
Score:
4
|
|
Weighted
Score:
6
|
|
Weighted
Score:
6
|
|
Weighted
Score:
8
|
|
| |
|
|
Win NT or 2000, 500MB free disk space, 64MB RAM. Pentium 133
or better. |
|
Windows 9x, NT 100MB disk space 64MB RAM. |
|
Win 95/NT 64MB RAM 100MB Disk Space. |
|
Win NT 4.0 or Windows 9x, Windows 2000, 64MB RAM, 70MB disk
space, Pentium Pro or better. |
|
| |
|
Technical
Support |
|
| |
|
Weight: 4 |
|
Weighted
Score:
24
|
|
Weighted
Score:
16
|
|
Weighted
Score:
24
|
|
Weighted
Score:
24
|
|
| |
|
|
Technical support is available by phone (9:00AM - 6:00PM) or
email. International Support is also available for Asia Pacific,
Europe, Middle East, and Africa. |
|
Phone support is available worldwide along with email and
and web support forms. |
|
Mercury Interactive has many different support programs
available. They are available by phone throughout the world with
varying hours. |
|
Product support available by phone between 9AM and 6PM EST.
Support questions can also be submitted through the company website.
The staff was knowledgeable, and returned calls in a timely
fashion. |
|
| |
|
Training
Available |
|
| |
|
Weight: 5 |
|
Weighted
Score:
64
|
|
Weighted
Score:
32
|
|
Weighted
Score:
64
|
|
Weighted
Score:
32
|
|
| |
|
|
Rational University provides training throughout the United
States and Canada. Classes not only focus on products but also the
principles of functional testing. |
|
Classes are available across the United Stated. Consulting
services are also available |
|
Mercury provides Partner-Certification programs, On-Site
Training, WorldWide Training along with offering its own class at
different sites throughout the U.S. |
|
Training is available onsite; consulting and partner
certification is also available. |
|
| |
|
|
|
| |
|
Reporting |
|
|
|
|
| |
|
Environment for retrieving and manipulating stored
reports. |
|
| |
|
Factor: Standard |
|
| |
|
Product Name |
Rational Performance
Studio
1.5 |
SilkTest
5.0.2 |
WinRunner
6.0 |
e-Test
Suite
4.2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Manufacturer |
Rational Software |
Segue |
Mercury
Interactive |
RSW Software, Inc. |
|
| |
|
|
Group Score:
88 out of 140
Total Score:
989 out of 1520 |
|
Group Score:
84 out of 140
Total Score:
845 out of 1520 |
|
Group Score:
60 out of 140
Total Score:
816 out of 1520 |
|
Group Score:
48 out of 140
Total Score:
736 out of 1520 |
|
| |
|
Log
Viewer |
|
| |
|
Weight: 4 |
|
Weighted
Score:
32
|
|
Weighted
Score:
24
|
|
Weighted
Score:
24
|
|
Weighted
Score:
16
|
|
| |
|
|
The log viewer can be set to display automatically after
test completion. Shows verification points included in the script as
well as Pass or Fail, time and date of test. |
|
The results file automatically displays after the completion
of the test cases. The results appear in a window within the
SilkTest application not as a separate application. |
|
All events can be viewed in the Test Results Viewer which
can be configured to display automatically upon test
completion. |
|
The log viewer is simple with few features. It gives the
user the ability to export the log to a text file. |
|
| |
|
Generates
Reports |
|
| |
|
Weight: 4 |
|
Weighted
Score:
24
|
|
Weighted
Score:
24
|
|
Weighted
Score:
8
|
|
Weighted
Score:
0
|
|
| |
|
|
Summary reports can be created using the Test Manager
product that's included with the product suite. The summary report
includes Project name, script name, and type and description. Defect
reports can be generated from the ClearQuest as long as the defects
were entered from the log viewer. |
|
Completion reports are generated to track the progress of a
test plan. After a test case is ran a report is automatically
generated that displays the results of the ran tests. Information
includes pass/fail, started and elapsed time. |
|
Results can be generated into a text-based report. Further
reporting requires the use of the Test Suite. |
|
No
reports are generated for results. All test information is written
to log files. Third party tools can be used. |
|
| |
|
Log
Filters |
|
| |
|
Weight: 3 |
|
Weighted
Score:
16
|
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
Weighted
Score:
8
|
|
| |
|
|
Filters can also be created using the Log Viewer. Included
is the ability to include event types. |
|
Filters can be set to provide as much or little information
as the user wants. Results files may also be merged with other
results files. |
|
The type of events to be shown in the results logs can be
selected from the Test Results viewer. |
|
Log options allow the user to report failures only or all
details. No other filtering is available. |
|
| |
|
Runtime
Message Logging |
|
| |
|
Weight: 4 |
|
Weighted
Score:
16
|
|
Weighted
Score:
24
|
|
Weighted
Score:
16
|
|
Weighted
Score:
24
|
|
| |
|
|
All messages are sent to log files. |
|
Messages are written automatically to the log file. Scripts
can also be modified to pass any information to the log file the
user may find useful. |
|
All events are written directly to the log files. |
|
During playback, if the script all messages can be viewed as
they occur, they are also written to the log file. |
|
| |
|
|
|
| |
|
Test Controller
Criteria |
|
|
|
|
| |
|
The
feature of the overall control criteria for test
execution. |
|
| |
|
Factor: Standard |
|
| |
|
Product Name |
Rational Performance
Studio
1.5 |
SilkTest
5.0.2 |
WinRunner
6.0 |
e-Test
Suite
4.2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Manufacturer |
Rational Software |
Segue |
Mercury
Interactive |
RSW Software, Inc. |
|
| |
|
|
Group Score:
97 out of 175
Total Score:
989 out of 1520 |
|
Group Score:
44 out of 175
Total Score:
845 out of 1520 |
|
Group Score:
73 out of 175
Total Score:
816 out of 1520 |
|
Group Score:
109 out of 175
Total Score:
736 out of 1520 |
|
| |
|
Runtime
Test Control |
|
| |
|
Weight: 3 |
|
Weighted
Score:
16
|
|
Weighted
Score:
8
|
|
Weighted
Score:
12
|
|
Weighted
Score:
16
|
|
| |
|
|
Many parameter options are available to control the playback
of the script. Some include command execution delay, error recovery,
logs, and diagnostic tools to be used. |
|
Runtime options available to control tests include Type of
Network, agent name, browser type and few others. |
|
Some of the options that can be set include checkpoint
timeouts, some error conditions, and speed of execution. |
|
Options that can be set to control playback include, but are
not limited to, error recovery, page timeout, and use of
DataBanks. |
|
| |
|
Supports
Forms |
|
| |
|
Weight: 2 |
|
Weighted
Score:
8
|
|
Weighted
Score:
6
|
|
Weighted
Score:
6
|
|
Weighted
Score:
6
|
|
| |
|
|
Yes, datapools can be used to generate information for a
more realistic test of form input. |
|
SilkTest provides support for forms. |
|
WinRunner provides support for forms. |
|
e-Test Suite provides support for forms. |
|
| |
|
Parameterization |
|
| |
|
Weight: 1 |
|
Weighted
Score:
5
|
|
Weighted
Score:
2
|
|
Weighted
Score:
3
|
|
Weighted
Score:
3
|
|
| |
|
|
User parameters such as login or registration information
can be either imported from a file or generated by Rational through
the use of Datapools. |
|
Parameterization is accomplished by adding statements to the
script that identify it as a data driven test and then specifying
the data type and data file to use. |
|
Tests can be parameterized by importing data from a file, a
database, or by using the DataDriver Wizard. |
|
DataBanks can be used to provide user parameters to make a
more realistic test. |
|
| |
|
Import
Results Files |
|
| |
|
Weight: 5 |
|
Weighted
Score:
48
|
|
Weighted
Score:
0
|
|
Weighted
Score:
0
|
|
Weighted
Score:
64
|
|
| |
|
|
Files can be imported to be used as verification points
called file comparison. This test can check that a file created
within the application contains the correct information. |
|
SilkTest does not allow the user to import results
files. |
|
Documentation mentions expected results files but does not
mention any way to import them. |
|
Using Databanks, the user can import not only results but
the input data also. |
|
| |
|
Database
Verification |
|
| |
|
Weight: 4 |
|
Weighted
Score:
8
|
|
Weighted
Score:
16
|
|
Weighted
Score:
40
|
|
Weighted
Score:
8
|
|
| |
|
|
No
specific functionality is available to facilitate this feature. A
script could be written to achieve this functionality. |
|
No
specific feature is supplied to test that a transaction was
successfully written to a database. It is possible to write a query
within the script to test this though. |
|
Database checkpoints can be added to the script to test that
a transaction was successfully entered into a database. |
|
No
specific feature is available to facilitate this feature. Can be
accomplished by writing a scriptlet. |
|
| |
|
Supports
Proxy Server Connections |
|
| |
|
Weight: 3 |
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
| |
|
|
Yes |
|
Yes |
|
Yes. |
|
Yes |
|
| |
|
|
|
| |
|
Tools and Options |
|
|
|
|
| |
|
The
regular utilities and configurable aspects of the product. |
|
| |
|
Factor: Standard |
|
| |
|
Product Name |
Rational Performance
Studio
1.5 |
SilkTest
5.0.2 |
WinRunner
6.0 |
e-Test
Suite
4.2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Manufacturer |
Rational Software |
Segue |
Mercury
Interactive |
RSW Software, Inc. |
|
| |
|
|
Group Score:
388 out of 580
Total Score:
989 out of 1520 |
|
Group Score:
380 out of 580
Total Score:
845 out of 1520 |
|
Group Score:
232 out of 580
Total Score:
816 out of 1520 |
|
Group Score:
240 out of 580
Total Score:
736 out of 1520 |
|
| |
|
Browser
Selection |
|
| |
|
Weight: 3 |
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
Weighted
Score:
0
|
|
Weighted
Score:
8
|
|
| |
|
|
The browser type can be selected from the GUI playback
menu. |
|
SilkTest allows the user to select a browser. |
|
Browser selection is not available. |
|
e-Test Suite provides the ability to generate a Netscape 4.0
or higher scripts. Scripts created are compatible for Microsoft
Internet Explorer by default. |
|
| |
|
Bug
Tracking |
|
| |
|
Weight: 5 |
|
Weighted
Score:
64
|
|
Weighted
Score:
48
|
|
Weighted
Score:
32
|
|
Weighted
Score:
0
|
|
| |
|
|
Errors or defects can be entered automatically from the log
viewer into the ClearQuest product. |
|
SilkTest integrates directly with SilkRadar to accommodate
bug tracking functionality. SilkRadar is purchased separately from
SilkTest. |
|
Test Director is needed to track defects. |
|
This functionality is not supported, but third-party tools
can be used if desired. |
|
| |
|
Data
Driven Tests |
|
| |
|
Weight: 5 |
|
Weighted
Score:
64
|
|
Weighted
Score:
48
|
|
Weighted
Score:
48
|
|
Weighted
Score:
64
|
|
| |
|
|
Data driven tests are supported, and files supplying test
and results data can be imported easily. |
|
Data driven tests are supported by SilkTest. |
|
The Data Driver Wizard can be used to parameterize the
scripts. Microsoft Query or Data Junction can also be used to import
data. |
|
DataBanks can be used to map external data sources to
facilitate data-driven tests. |
|
| |
|
Error
Recovery Feature |
|
| |
|
Weight: 5 |
|
Weighted
Score:
48
|
|
Weighted
Score:
80
|
|
Weighted
Score:
48
|
|
Weighted
Score:
64
|
|
| |
|
|
Error recovery option can be set for Script command failures
or verification point failure, also the user can set recovery for
unexpected windows that appear. |
|
SilkTest error recovery feature returns the application to a
base-state before each test in a test plan is executed. Which makes
running multiple tests much easier and accurate. |
|
Exceptions are created under the tools menu. They direct
WinRunner on how to handle execution when an error occurs. A number
of exceptions for web-based applications are provided. |
|
Error recovery options are set under the Options menu in
e-Tester. Provides a table that allows the user to select what
actions to take when an error occurs. |
|
| |
|
Runs
Unattended |
|
| |
|
Weight: 4 |
|
Weighted
Score:
0
|
|
Weighted
Score:
32
|
|
Weighted
Score:
8
|
|
Weighted
Score:
24
|
|
| |
|
|
This functionality is not available for functional
testing. |
|
Though there is no scheduler to automatically run tests at a
specified time. The error recovery feature allows tests to be
started at the end of a day and ran over night by resetting the
application to a base state after each execution. |
|
This functionality is available using the Test
Director. |
|
This functionality is provided with e-Monitor |
|
| |
|
Source
Code Control Systems |
|
| |
|
Weight: 5 |
|
Weighted
Score:
32
|
|
Weighted
Score:
32
|
|
Weighted
Score:
16
|
|
Weighted
Score:
16
|
|
| |
|
|
Rational Perfmance Studio provides some useful
functinoality, but is not directly integrated with a source code
control system. |
|
SilkTest integrates with PVCS. |
|
This functionality is not provided, but a third-party tool
could be used if desired. |
|
This functionality is not provided, but a third-party tool
could be used if desired. |
|
| |
|
Test
Plan Integration |
|
| |
|
Weight: 5 |
|
Weighted
Score:
64
|
|
Weighted
Score:
80
|
|
Weighted
Score:
48
|
|
Weighted
Score:
16
|
|
| |
|
|
TestManager tool helps the user plan and synchronize tests.
Included with the PerformanceStudio 1.5 |
|
This functionality is built in to SilkTest and a key feature
in its architecture. |
|
WinRunner can integrate directly with the Test Director
which is Mercury's Test Plan Management system. |
|
This functionality is not included but the product is
designed to integrate with third party tools. |
|
| |
|
Link
Checker |
|
| |
|
Weight: 5 |
|
Weighted
Score:
64
|
|
Weighted
Score:
48
|
|
Weighted
Score:
32
|
|
Weighted
Score:
48
|
|
| |
|
|
Rational SiteChecker is included with the PerformanceStudio;
it is a comprehensive tool that can check the integrity of your
website links internally and externally. |
|
This functionality is included with a new feature called
LinkTester |
|
To
check if links are valid, a checkpoint must be inserted into the
script. They can test individual links or all links within a
frame. |
|
Resource validation can run automatically after the
completion of the test. Results can be exported to a text
file. |
|
| |
|
Generation
of Test Data |
|
| |
|
Weight: 4 |
|
Weighted
Score:
40
|
|
Weighted
Score:
0
|
|
Weighted
Score:
0
|
|
Weighted
Score:
0
|
|
| |
|
|
Datapools are used to create user data to simulate a more
realistic test scenario. |
|
The product does not support the generation of test
data. |
|
Data can be mapped using the Data Driver Wizard but not
created. |
|
The product does not support the generation of test data,
but third party tools can be used if desired. |
|
| |
|
|
|
| |
|
Security |
|
|
|
|
| |
|
Security criteria may include, but is not limited to,
standards such as SSL and SET, protection of administrative
interfaces, user and administrator access, key management (PKI),
encryption, authentication, firewalls, virus protection, and other
various security tools and features. |
|
| |
|
Factor: Standard |
|
| |
|
Product Name |
Rational Performance
Studio
1.5 |
SilkTest
5.0.2 |
WinRunner
6.0 |
e-Test
Suite
4.2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
|
Manufacturer |
Rational Software |
Segue |
Mercury
Interactive |
RSW Software, Inc. |
|
| |
|
|
Group Score:
60 out of 100
Total Score:
989 out of 1520 |
|
Group Score:
60 out of 100
Total Score:
845 out of 1520 |
|
Group Score:
60 out of 100
Total Score:
816 out of 1520 |
|
Group Score:
60 out of 100
Total Score:
736 out of 1520 |
|
| |
|
Supports
SSL and SHTTP |
|
| |
|
Weight: 5 |
|
Weighted
Score:
48
|
|
Weighted
Score:
48
|
|
Weighted
Score:
48
|
|
Weighted
Score:
48
|
|
| |
|
|
The product supports SSL and SHTTP. |
|
The product supports SSL and SHTTP. |
|
The product supports SSL and SHTTP. |
|
The product supports SSL and SHTTP. |
|
| |
|
Supports
Username and Password Login |
|
| |
|
Weight: 3 |
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
Weighted
Score:
12
|
|
| |
|
|
The product supports username and password login and will
automatically accept and process user verification
requests. |
|
The product supports username and password login and will
automatically accept and process user verification
requests. |
|
The product supports username and password login and will
automatically accept and process user verification
requests. |
|
The product supports username and password login and will
automatically accept and process user verification
requests. |
|
| |
|
|
|
