Platform design#

This section contains a description of the TAO platform from architectural point of view.

The logical data flow of the TAO platform is provided in the next figure:

Logical data flow of the TAO platform

From logical data flow point of view we can distinguish the components:

  • Resource Catalog - the repository from where the user can select 'processing components' (e.g. modules from SNAP, OTB, etc.) and 'data products' as input data for processing workflow. The access to 'data products' is provided via 'data sources' which represents an abstractization of a way to connect to a live set of data. Physically, the files that make up a 'data product' can be accessed locally (local file system) or remotely through the web services provided by known cloud platforms (SciHub, Amazon WS).

  • Orchestrator - play the role to decompose a workflow job into tasks and to send them for execution to the Cluster Manager that distribute their execution over the eligible nodes.

  • Cluster Manager - coordinate the tasks execution in a distributed way and manage the processing resources.

Platform use cases#

There are two user roles, playing as:

  • Public User - type of user who exploit the TAO platform functionalities

  • Administrator - type of user in charge with platform configuration and administration

Platform interfaces#

The platform is interfacing with:

  • Data products - through data sources, all internal parts of a data product (images themselves, description files with metadata) can be read and interpreted

  • Rendering system - for displaying the input data products and also the final results the GeoStorm system has been chosen

  • External processing platforms - the communication with external processing platforms is made using the WPS protocol implemented as client (for making connection to other systems) as well as server (to accept connections from other systems)

Software validation and verification plan#

Synthesis of requirements#

The V&V activities are targeting the requirements found in the table below:

ID Requirement

Validation/

Verification method

Functional requirements

Resource Administration [core]

RQ-RAD-01 It shall be possible for the user (Administrator or Public User) to add, edit or delete resources into system. Test
RQ-RAD-02 A resource can be deleted from system if there is no reference to it. Test
RQ-RAD-03 Resources added by Administrator can be used by all users. Test
RQ-RAD-04 The flush operation of a node should offer the possibility to move the entire execution environment to another eligible node. Test

User Workspace [core]

RQ-UWS-01 The WM shall allow the definition and management of workflows. Test
RQ-UWS-02 The WM shall allow the link between two components only if the output of one component matches the input requirements of the other component. Test
RQ-UWS-03 The WM shall be able to convert the drawn workflows into jobs to be executed. Test
RQ-UWS-04 The WM shall be able to find the eligible nodes for the corresponding tasks within a workflow. Test
RQ-UWS-05 The WM shall be able to find the eligible nodes for the corresponding tasks within a workflow. Test

Processing Component Integration [core]

RQ-PCI-01 The framework shall be able to integrate processing components (i.e. executables) written in different programming languages. Analysis
RQ-PCI-02 The framework shall integrate at the same time processing components written for only one targeted operating system. Analysis
RQ-PCI-03 Processing components shall be exposed to the framework in the same way (i.e. a single interface), regardless of the programming language in which they were written. Inspection
RQ-PCI-04 The wrapping interface for a processing component shall be able to pass to the processing module the input from the workflow context. Test
RQ-PCI-05 The wrapping interface for a processing component shall be able to invoke the processing module with the appropriate parameters. Test
RQ-PCI-06 The wrapping interface for a processing component shall be able to report the execution progress and status back to the workflow manager. Test
RQ-PCI-07 The wrapping interface for a processing component shall be able to integrate the results of the processing component back into the parent workflow. Test
RQ-PCI-08 The framework shall provide to each integrated processing component the appropriate set of runtime dependencies. Inspection
RQ-PCI-09 Processing components executing on the same processing node (i.e. machine) shall have not interfere with each other and shall be executed inside dedicated containers. Analysis
RQ-PCI-10 If multiple processing components (i.e. belonging to the same toolbox) share the same runtime dependencies, they shall be executed inside the same execution container. Analysis

Data Access [core]

RQ-DAC-01 The framework shall allow for retrieving EO and ancillary data from both local and remote repositories. Test
RQ-DAC-02 A local data repository shall consist in a combination of a database and file system. Metadata (i.e. data descriptors) shall be stored in the database, while data themselves shall be stored in file system. Analysis
RQ-DAC-03 The framework shall abstract the source of data and the data. All registered data sources shall be queried in the same way and they shall produce compatible results. Inspection
RQ-DAC-04 A data source component shall allow the API or the user to provide credentials needed to connect to a repository. Inspection
RQ-DAC-05 A data source component shall allow queries that return one or more EO data products. Inspection
RQ-DAC-06 A remote data source component shall take into account the users quotas before actually retrieving remote products. Test
RQ-DAC-07 A data source component shall allow specifying if the returned results shall be accessible only by the calling API or user, or they shall be available to all the users of the framework instance. Test
RQ-DAC-08 The framework shall provide as default several data sources for radar and optical data. The data products returned by these sources will be accessible by all users of the framework instance. Analysis
RQ-DAC-09 The local data repository can support retention policies. Analysis

Workflow Execution Management [core]

RQ-WEM-01 The WEM shall be able to manage the execution of workflows as produced by the Workflow Management feature. Analysis
RQ-WEM-02 The WEM shall be able to execute a workflow, to return the execution status of any running processing module (component) involved in a workflow and to allow the return of results (if persisted). Test
RQ-WEM-03 The WEM shall be able to monitor the execution of all workflow jobs. Test
RQ-WEM-04 The WEM can support custom allocation of resources, in terms of memory, CPU, disk or network, for a given execution. Analysis
RQ-WEM-05 The WEM can support the reservation of resources into a node, in terms of memory, CPU, disk or network, for a given execution, dedicated to a given user. Analysis
RQ-WEM-06 The WEM should support an interface to provide status and usage information to the other components via a given API. Inspection
RQ-WEM-07 The WEM should be able to block a task execution on request from the Administrator and release the reserved resources. Test
RQ-WEM-08 The WEM can be able to pause activity job execution on request of the user, releasing partially or totally the reserved resources, and then resume it later. Test

Resource Management [additional]

RQa-RMG-01 The platform shall provide a catalog component. Analysis
RQa-RMG-02 The catalog component shall respect standards such as CWS, OpenSearch, ISO 119115, WCS. Review
RQa-RMG-03 The catalog component should reference information from external data provider. Inspection
RQa-RMG-04 The catalog component should harvest data (at least metadata) from external data provider. Inspection
RQa-RMG-05 The platform shall authenticates users before allowing access to its services and data. Test
RQa-RMG-06 The platform shall allow setting authorizations to users, groups on access to services and data. Test
RQa-RMG-07 Authenticated users can only access services and data they have authorizations to use. Test
RQa-RMG-08 The platform shall be able to limit access to its services (processing, disk space, etc.) accordingly to the quota set by user or group of users. Test

User Workspace [additional]

RQa-UWS-01 A user shall be able to access the resource catalog based on the given access control rights. Test
RQa-UWS-02 Data held by the platform shall be displayed on a map-centered graphical interface. Test
RQa-UWS-03 Input, output and intermediate (georeferenced) data, if marked as public or shareable, shall be displayable on a map. Test

Workflow Execution Management [additional]

RQa-WEM-01 Through WEM module, it shall be possible to schedule the execution of a job. Test
RQa-WEM-02 Execution of a job shall be possible to be triggered by an event. Test
Interface requirements
RQ-INT-01 Communication between the TAO platform services (components) and external interfaces shall be done via REST, SOAP or plain HTTP requests over HTTP. Review
RQ-INT-02 Communication messages between platform internal components written in the Java programming language shall be preferably done using JRMP or plain TCP sockets. Review
RQ-INT-03 Communication messages between platform internal components written in the C/C++ programming language shall be preferably done using SSH or plain TCP sockets. Review
RQ-INT-04 Communication messages between platform internal components written in the Java programming and components written in C/C++ programming language shall be done using JNI or JNA. Review
RQ-INT-05 The TAO platform will have a dedicated network file system area accessible by all the TAO components for the data sharing mechanism. Review
Resource requirements

Hardware Resource

RQ-HWR-01

The master node (or the workstation – in the case of a local service) shall have the following minimum hardware configuration:

  • Processor: x64 architecture, quad core, 2.0 GHz

  • Memory: 16GB RAM DDR3

  • Storage: 200GB

Analysis
RQ-HWR-02

Each execution node shall have at minimum the following hardware configuration:

  • Processor: x64 architecture, dual core (with SMT/HT) or quad core, 2.0 GHz

  • Memory: 16GB RAM DDR3

  • Storage: 200GB for processing nodes, 1TB for catalog nodes

Analysis

Software Resource

RQ-SWR-01 The target operating system for TAO as a platform (cluster mode) shall be CentOS 7. Analysis
RQ-SWR-02 The target operating systems for TAO as a service (desktop mode) shall be at least Microsoft Windows 7 and Ubuntu Linux 14. Analysis
RQ-SWR-03 The minimum Java Runtime shall be JSE 1.8.0_102. Analysis
RQ-SWR-04

The minimum version for the software toolboxes used by the TAO platform shall be the one from the table:

Software Minimal Version Used in
Orfeo Toolbox 7.3.0 Processing Components
SNAP 9.0.0 Processing Components
GDAL 3.2.2 Processing Components
jsPlumb Workflow Management
Torque Cluster Manager
Docker Processing Components Deployment
OpenLayers 3 Data Visualization
Review
Design requirements
RQ-DES-01 The TAO design shall be performed using UML. Review
RQ-DES-02 The TAO design shall include, at minimum, use cases and interaction diagrams for all the software features. Review
RQ-DES-03 The TAO components should be exposed to the framework via an interface. Inspection
RQ-DES-04 The implementation of a specific TAO component should not be visible to other components (i.e. other components should not care about / depend on the implementation of other components). Inspection
RQ-DES-05 The implementation language for the framework components development will be Java. Review
RQ-DES-06 For those components that make use of third party libraries / applications that are written in programming languages other than Java, specific Java wrappers for these libraries / applications will be created (if not existing already). Inspection
Security and Privacy requirements
RQ-SEP-01 The platform shall be usable by a user by means of an account with a password. Test
RQ-SEP-02 The users shall be able to change the password of their respective accounts by themselves. Test
RQ-SEP-03 The user-defined artifacts, together with their results, shall be by default private (i.e. visible only to their author). Test
RQ-SEP-04 The platform shall allow a user to share one or more of its artifacts and results with other users. Test
RQ-SEP-05 An access control mechanism shall be implemented so that certain features can be used only by an administrator. Test
RQ-SEP-06 The access control mechanism shall be fine-grained so that certain component attributes or items can be read-only for (i.e. can be used by) regular users and can be edited only by the administrator. Test
Portability requirements
RQ-POR-01 When run as a service (desktop mode), the framework shall be able to run on different operating systems, according to [RQ-SWR-02]. Analysis
RQ-POR-02 When run as a platform (cluster mode), the framework shall be able to run on the operating system(s) specified in [RQ-SWR-01]. Analysis
Software Quality requirements
RQ-SQA-01 When run in cluster mode, the framework shall be able to scale horizontally (i.e. increasing the number of processing nodes) by providing a mechanism for automatic configuration of nodes. Inspection
RQ-SQA-02 The number of workflows that can be executed in parallel shall be at most two times the number of processing nodes. Inspection
Data definition and Database requirements
RQ-DDD-01 The input data (either EO products or ancillary data) shall be limited to what is currently supported by the integrated toolboxes (OTB and SNAP). Inspection
RQ-DDD-02 The processing component wrapper definitions shall be stored in a local relational database (LRDB). Inspection
RQ-DDD-03 The workflow definitions shall be stored in LRDB. Inspection
RQ-DDD-04 All the workflow states (i.e. execution step at a given moment) shall be persisted in LRDB. Inspection
RQ-DDD-05 The LRDB shall be in at least third normal form (i.e. shall ensure referential integrity). Inspection
RQ-DDD-06 In addition to the above requirements, the LRDB shall hold information at least for the following: user accounts, user workspaces, catalog item metadata. Inspection
RQ-DDD-07 The framework shall be able to export workflow and processing components definition. The export format will be XML. Inspection
RQ-DDD-08 The framework shall be able to import workflow and processing components definition from XML files. Inspection

Table 3‑1 : Summary of SRS requirements

Test case Pass/Fail criteria#

The item pass/fail criterion depends on the validation/verification method for the item. The verification method is specified in SRS document [A1], also in table Table 3‑1.

Possible verification methods are:

  • Test verification method [T] - The method is referred as “Test” when requirements have to be validated by measuring product performance and function under various simulated environments. These measurements may require the use of special equipment, instrumentation and simulation techniques. Established principles and procedures shall be used to determine conformance to requirements.
  • Inspection verification method [I] - The method is referred as “Inspection” when verification is achieved by visual determination of physical characteristics (such as construction features, hardware conformance to document drawing or workmanship requirements).

  • Analysis verification method [A] - The method is as “Analysis” when verification is achieved by performing theoretical or empirical evaluation by accepted techniques. The analytical techniques shall be selected amongst systematic, statistical and qualitative design analysis, modelling and computational simulation. Verification by similarity is considered part of Analysis. It shall be applied if it can be shown that the article under verification is similar to another article that has already been verified to equivalent or more stringent requirements.

  • Review of design verification [R] - The method is referred as “Review” when verification is achieved by validation of records or by evidence of validated design documents or when approved design reports, technical descriptions, engineering drawings unambiguously show that the requirement is met.

All tests have a number of steps. The general criteria is:

  • a test passed if the expected results are achieved for all the steps of the procedure related to this test.

  • a test failed if the expected results are not achieved for at least one step of the procedure related to this test.

Software validation process#

The software validation process confirms that the functional requirements are correctly and completely implemented in the final TAO platform.

The validation procedures with respect to the functional specifications are run to validate the developed TAO platform.

Validation process implementation#

The TAO platform was validated by testing against functional requirements and against the user requirements.

Validation activities#

The validation activities described in this section will be used to validate the developed processing system with respect to the functional specifications and user requirements.

Validation tasks identification#

The validation activities were performed by test (running test cases) and when validation by test was not applicable, the validation was performed by analysis, inspection, or review of design.

Software verification process#

The software verification process confirms that adequate specifications and inputs exist for every activity and that the outputs of the activities are correct and consistent with the specifications and input.

Verification process implementation#

Depending on the item being the subject of verification, different methods and techniques are described to be used during the verification process. All the requirements that are verified are checked that they are consistent and verifiable. The requirements tested in the verification process are the interface, resource, design, portability, software quality and database requirements.

Verification activities#

Verification of software requirements#

The verification of the software requirements implies inspecting the SoW and the SRS documents.

Along with inspecting the traceability between software related system requirements from SoW and software related system requirements, this activity ensures that the SRS contains clear, consistent and verifiable requirements regarding:

  • the operational demo environment;

  • the external interfaces;

  • necessary configuration data;

  • processing resources (memory, CPU and HDD) allocation policy;

  • the scenarios chosen for demonstration.

Verification of the software architectural design#

This verification activity will ensure:

  • the resource management, the integration of processing components and the orchestration of scientific workflows takes into consideration the SRS requirements;

  • the platform design respects the SRS requirements and it provides internal consistency between the software components;

  • traceability between the technical requirements and the software components design requirements.

Verification of code#

This verification activity will ensure that:

  • the code is externally consistent with the requirements and the design of the software items;

  • the code is traceable to design and requirements, testable, correct, and in conformity to software requirements and coding standards.

Verification of software unit testing#

The unit test plans and strategy will be verified that they are:

  • consistent with the design and requirements;

  • traceable to software components, design and code

  • the results conform to the expected results and the results, plans, test data, test cases and procedures are documented and under configuration control.

Verification of software documentation#

This activity will ensure that:

  • the software documentation is complete, consistent and configuration managed;

  • the technical requirements related to software documentation are respected;

  • the documentation management follows the specified procedures.

Software validation and verification report#

The SVTR document will contain the results of the tests that were executed during the validation and verification process, following the procedures defined for the software platform validation and verification.

Platform configuration settings#

This section describes the configurable platform components and the necessary settings for each of the components, taking into consideration the platform that is being used for validation.

Configurable platform components#

Hardware platform#

The demo platform hardware specifications are documented in table below:

Server Virtual Machine
Processors 8 vCPU x64
Memory 16 GB RAM
Disk Drives 200 GB SSD, 1 TB HDD attached volume

Table 4‑1 : Hardware specification

Software processing system configuration#

Operating system#

The operating system used by the TAO platform is CentOS 7.0. There are no particular operating system settings to be applied. The operating system shall be installed using default installation parameters and packages.

Database server#

The TAO platform uses the PostgreSQL 11 (or higher) database server. The database server shall use the default CentOS package settings.

Processing components#

The TAO processors require a set of parameters that configure their operation. The configuration parameters will be loaded from a file located in the standard operating system location.

Software Delivery and Verification Process#

This process prepares the software platform for delivery and installation in its demo environment. It is completed with an AR, with the SVVP as input. During the AR, the software platform is evaluated in its demonstration environment.

Delivery process VERIFICATION#

The procedures from section 6.2 will be used to verify the software platform delivery in its demonstration environment.

Installation procedure and activities validation#

The procedures from section 6.2 will be used to verify the software platform installation in its demonstration environment.

Verification process specification#

This section defines the verification process that will be followed in order to ensure the acceptance by ESA of the final TAO platform. The verification process will make sure that the developed platform during the TAO project complies with the requirements specified in the SRS. The verification tests will verify the behavior of the TAO platform for EO input data products and their processing within the workflow.

The verification tests should be executed during the Acceptance Review meeting of the TAO project.

Verification test approach#

The user requirements mentioned in SRS document are tested for the acceptance of the TAO platform.

The item being tested, the verification criteria, the necessary inputs, the environmental needs, the expected results are defined in the associated test case specifications. The test cases will be run in the conditions defined by the test case specifications and following the steps from the test procedures.

The following items must be delivered before verification testing begins:

  • Documents: SRS, SVVP;

  • Verification test input data;

  • TAO platform to be tested.

When the verification testing is finished, an SVTR will be generated.

Verification criteria#

Every test case describes the verification criteria that should be met to pass that specific test in the “Pass/fail criteria” section of the test case specification.

Tasks identification#

The following tasks are necessary in order to prepare and perform the verification tests:

  • Designing the verification tests;

  • Ensuring that all environmental needs are satisfied for the verification tests;

  • Performing the verification tests.

Software Verification Test Report#

The results of the tests that will be run for the verification of the TAO demonstration platform, following the procedures described in this SVVP, will be documented in the TAO SVVTR.

Appendix 1 - Software verification test cases#

Test case specifications#

This chapter contains a specification for each test case that has to be tested against the SRS and for each of these test cases, the following information is specified:

Test case identifier: Title The test case unique identifier followed by a title
Test item A short description of the test case purpose
Covered requirements Covered specification requirements
Input specification The inputs to execute the test case
Expected results The expected results
Pass/fail criteria The criteria to decide whether the test has passed or failed
Environmental needs The environment used for the execution of the test case. This can include the configuration and the setup of the facility used to execute the test case as well as the utilization of any special test equipment; the configuration of the software utilized to support the test conduction;
Dependencies List of all the test cases to be executed before this test case

Table 6‑1 : Test case organization

Authentication#

TC-AUTH-01: login on platform
Test item Test to show the user can login into platform using his credentials
Covered requirements RQa-RMG-05, RQ-SEP-01
Input specification User login into platform with username / password
Expected results Successful login
Pass/fail criteria User is active, the platform recognize the credentials and create the session for user
Environmental needs TAO platform is up and running
Dependencies N/A
TC-AUTH-02: logout from platform
Test item Test to show the user can logout from platform
Covered requirements RQa-RMG-05
Input specification

User's session is not expired

User logout from platform

Expected results Successful logout
Pass/fail criteria The user session is closed
Environmental needs TAO platform is up and running
Dependencies TC-AUTH-01: login on platform
TC-AUTH-03a: reset user password - request
Test item Verifies if the user can request to reset his password
Covered requirements RQ-SEP-02
Input specification Logged user request the reset of his password
Expected results An email with a reset link (with token) is successfully sent
Pass/fail criteria User reset his password if has a valid email address
Environmental needs TAO platform is up and running
Dependencies TC-AUTH-01: login on platform
TC-AUTH-03b: reset user password - confirmation
Test item Verifies if the user reset password can be completed as result of confirmation
Covered requirements RQ-SEP-02
Input specification User confirm the reset action by following the sent link (with token) and input the new password
Expected results The user can login with the new password
Pass/fail criteria User's password is reset if the received token is valid and new password meets the strength requirements
Environmental needs TAO platform is up and running
Dependencies

TC-AUTH-01: login on platform

TC-AUTH-03: reset user password - request

Authorization#

TC-AUTZ-01: access to Administration module
Test item Verifies if the user is not allowed to access the Administration module of platform
Covered requirements RQa-RMG-06, RQa-RMG-07, RQ-SEP-05
Input specification A public user is logged into platform.
Expected results User cannot access the Administration module
Pass/fail criteria The [Authorization] module filter access based on user permissions
Environmental needs TAO platform is up and running
Dependencies TC-AUTH-01: login on platform
TC-AUTZ-02: access to User Catalog
Test item Verifies if the user have access only to his own User Catalog
Covered requirements RQa-UWS-01, RQ-SEP-03
Input specification A public user is logged into platform.
Expected results The user has access only to his own User Catalog
Pass/fail criteria The [Authorization] module filter access based on user permissions
Environmental needs TAO platform is up and running
Dependencies TC-AUTH-01: login on platform

User Management#

TC-USER-01: add user
Test item Verifies if an Administrator can add an user
Covered requirements RQ-RAD-01
Input specification Administrator add an user
Expected results User is saved into DB
Pass/fail criteria User becomes available in the users list
Environmental needs TAO platform is up and running
Dependencies TC-AUTH-01: login on platform
TC-USER-02: define user quota
Test item Verifies if the Administrator can define user quota
Covered requirements RQ-RAD-01
Input specification Administrator select an user and change his quota
Expected results User quota is saved into DB
Pass/fail criteria The new user quota is displayed
Environmental needs TAO platform is up and running
Dependencies TC-AUTH-01: login on platform
TC-USER-03: remove user
Test item Verifies if the Administrator can remove an user
Covered requirements RQ-RAD-01
Input specification Administrator remove the selected user
Expected results User's session is closed, user is deactivated
Pass/fail criteria

The user cannot log in anymore

The Administrator account cannot be removed

Environmental needs TAO platform is up and running
Dependencies TC-AUTH-01: login on platform
TC-USER-04: edit user information
Test item Verifies if the user can edit the information of his account
Covered requirements RQ-RAD-01, RQa-RMG-06
Input specification User edit his account information
Expected results Information is saved into DB
Pass/fail criteria User's new information is displayed
Environmental needs TAO platform is up and running
Dependencies TC-AUTH-01: login on platform

Processing Components Management#

TC-COMP-01: CRUD a PC
Test item Verifies if the user can create/read/update/delete (CRUD) a PC
Covered requirements RQ-RAD-01, RQ-RAD-02, RQ-RAD-03
Input specification User make the action CRUD against a PC
Expected results PC descriptor is saved into DB
Pass/fail criteria

PC is available in Resource/User Catalog

PC is deployed on Docker Registry node

Deletion failed if there is a reference to PC

Environmental needs TAO platform is up and running
Dependencies
TC-COMP-02: attach a DS to a PC
Test item Test shows that a data source (DS) can be attached to a processing component (PC)
Covered requirements RQ-DAC-01
Input specification Within a workflow definition the user select a data source to be attached to a PC
Expected results

The link between DS and PC is saved.

Workflow definition is still coherent.

Pass/fail criteria Data products (DP) retrieved using DS are compatible with PC's data input
Environmental needs TAO platform is up and running
Dependencies

Data Sources Management#

TC-DSRC-01: edit DS parameters
Test item Verifies if a user can modify a DS parameters
Covered requirements RQ-DAC-07
Input specification User modify parameters of selected DS
Expected results DS parameters are saved into DB
Pass/fail criteria The new parameters can be viewed when inspect the DS
Environmental needs TAO platform is up and running
Dependencies

User Catalog Management#

TC-UCAT-01: CRUD an user data product
Test item Verifies if the user can create/read/update/delete a DP into User Catalog.
Covered requirements RQ-RAD-01, RQ-RAD-02, RQa-RMG-08, RQ-SEP-03
Input specification User perform a CRUD operation against a DP from his User Catalog
Expected results

DP descriptor is saved into DB

DP files are present into user's workspace

Pass/fail criteria

User quota allows the DP to be added

DP is available into User Catalog

Deletion failed if there is a reference to DP

Environmental needs TAO platform is up and running
Dependencies
TC-UCAT-02: share a private resource
Test item After sharing (i.e. making public) a resource from User Catalog, verifies if is visible into Resource Catalog
Covered requirements RQ-SEP-04
Input specification A private resource from user's own User Catalog is declared available for public use
Expected results The shared resource is accessible from Resource Catalog
Pass/fail criteria

The [Resource Catalog] is working properly

There is not enough quota on Resource Catalog

Environmental needs TAO platform is up and running
Dependencies

Topology#

TC-TOPO-01: add an execution node
Test item Verifies if the Administrator can add an execution node into Resource Catalog
Covered requirements RQ-RAD-01
Input specification In configuration phase, the Administrator add new nodes to be used for workflow execution
Expected results New node is present into Resource Catalog
Pass/fail criteria The new node is configured and has installed the TAO environment
Environmental needs TAO platform is up and running
Dependencies
TC-TOPO-02: delete an execution node
Test item Verifies if the Administrator can delete an execution node from Resource Catalog
Covered requirements RQ-RAD-02
Input specification The Administrator delete the node from Resource Catalog.
Expected results

If there are running tasks on node, these are relocated.

The TAO environment is un-deployed from node

The node is not present into Resource Catalog

Pass/fail criteria

All current jobs are still executed on existing nodes

There must be no reference to node

Environmental needs TAO platform is up and running
Dependencies
TC-TOPO-03a: edit an execution node - edit node information
Test item Verifies if the Administrator can change node's information
Covered requirements RQ-RAD-01
Input specification

The Administrator change node information, such as:

  • IP address

  • username

  • password

Expected results The new configuration of node is visible into Resource Catalog
Pass/fail criteria The node is available to run tasks
Environmental needs TAO platform is up and running
Dependencies
TC-TOPO-03b: edit an execution node - edit PC
Test item Verifies if the Administrator can modify an deployed PC on node
Covered requirements RQ-RAD-01
Input specification The Administrator modify a PC from an execution node
Expected results The new configuration of node is saved in Resource Catalog
Pass/fail criteria The PC from execution node is coherent and can be used to execute tasks
Environmental needs TAO platform is up and running
Dependencies
TC-TOPO-04: deploy a PC on an execution node
Test item Verifies if the Administrator can deploy a PC on an execution node
Covered requirements RQ-RAD-04
Input specification In configuration phase, the Administrator deploy PC on nodes presents into Resource Catalog. The deployment process suppose first the wrapper creation (packaging) and then the use of this wrapper by platform.
Expected results In Resource Catalog, the PC is available to run related tasks
Pass/fail criteria The node is eligible to run tasks related to PC
Environmental needs TAO platform is up and running
Dependencies
TC-TOPO-05: remove a PC from an execution node
Test item Verifies if the Administrator can remove a PC from an execution node
Covered requirements RQ-RAD-02
Input specification The Administrator remove a PC from an execution node
Expected results

If there are running tasks by PC, these are relocated.

The node is not eligible to execute related tasks to the PC that has been removed

Pass/fail criteria All current jobs are still executed on nodes
Environmental needs TAO platform is up and running
Dependencies
TC-TOPO-06: flush an execution node
Test item Verifies if the Administrator can flush an execution node
Covered requirements RQ-RAD-04
Input specification

The flush node operation implies the relocation of all running tasks

The Administrator flush an execution node

Expected results

The running tasks and related DP are relocated to another eligible node.

In Resource Catalog, the node is free, has no running tasks

Pass/fail criteria All current jobs are still executed on nodes
Environmental needs TAO platform is up and running
Dependencies
TC-TOPO-07: find an eligible execution node
Test item Test shows how the platform can find an eligible execution node
Covered requirements RQ-RAD-01
Input specification When the tasks are need to be relocated, an eligible node with the corresponding PC is searched
Expected results An eligible node is found
Pass/fail criteria The found node has the corresponding PC deployed
Environmental needs TAO platform is up and running
Dependencies

Workflow Management#

TC-FLOW-01: add a new workflow
Test item Verifies if an user can add a workflow into platform
Covered requirements RQ-RAD-01, RQ-UWS-01, RQa-RMG-08
Input specification The main purpose of TAO platform is construction of scientific workflows to process data products. Once the platform is configured, any user can build its workflows by chaining PC and continuity conditions.
Expected results The workflow is saved into User Catalog
Pass/fail criteria

The user quota allows the workflow definition

The workflow is present in User Catalog, ready to be executed

Environmental needs TAO platform is up and running
Dependencies
TC-FLOW-02: clone a workflow
Test item Verifies if an user can clone a workflow
Covered requirements RQ-UWS-01
Input specification A workflow from Resource Catalog can be re-used by any user through cloning operation. A workflow from User Catalog can be cloned only by its owner.
Expected results The workflow is saved into User Catalog having the DS without any credentials
Pass/fail criteria The workflow is present in User Catalog, ready to be executed
Environmental needs TAO platform is up and running
Dependencies
TC-FLOW-03: deactivate a workflow
Test item Verifies if an user or the Administrator can deactivate a workflow
Covered requirements RQ-UWS-01
Input specification An workflow is not deleted but deactivated. The Administrator deactivate workflows from Resource Catalog, while an user can deactivate his workflows from User Catalog
Expected results The workflow is marked as deactivated into Resource / User Catalog
Pass/fail criteria Deactivated workflows cannot be executed
Environmental needs TAO platform is up and running
Dependencies
TC-FLOW-04: publish a workflow
Test item Verifies if an user can publish a workflow
Covered requirements RQ-UWS-01
Input specification An user can publish its own workflow into Resource Catalog to be public available. Is the workflow cloning operation from User Catalog to Resource Catalog
Expected results The workflow is saved into Resource Catalog having the DS without any credentials
Pass/fail criteria The workflow is present in Resource Catalog, ready to be executed
Environmental needs TAO platform is up and running
Dependencies
TC-FLOW-05a: edit a workflow - add a PC
Test item Verifies if the user can add a PC to a workflow
Covered requirements RQ-UWS-02
Input specification Adding a PC to a workflow is permitted only if can be linked with adjacent PCs
Expected results The PC is saved into workflow definition
Pass/fail criteria The workflow is coherent
Environmental needs TAO platform is up and running
Dependencies
TC-FLOW-05b: edit a workflow - edit PC execution parameters
Test item Verifies if the user can configure execution parameters of a PC when is added into a workflow
Covered requirements RQ-UWS-02, RQ-SEP-06
Input specification Before a PC is added into a workflow, it should be configured from execution point of view. The user can edit only certain PC's parameters configured by Administrator to be public editable.
Expected results The configured PC is saved into workflow definition
Pass/fail criteria The PC parameters are valid
Environmental needs TAO platform is up and running
Dependencies
TC-FLOW-05c: edit a workflow - remove a PC
Test item Verifies if the user can remove a PC from a workflow definition
Covered requirements RQ-UWS-02
Input specification Remove a PC from a workflow definition is permitted only if the remaining adjacent PC can be linked
Expected results The workflow definition is saved
Pass/fail criteria The workflow is coherent
Environmental needs TAO platform is up and running
Dependencies

Dashboard#

TC-DASH-01: view currently used resources
Test item Verifies if the user can visualize the used resources
Covered requirements RQ-WEM-03
Input specification

User input the criteria to see :

  • a synthesis of used resources from all nodes

  • used resources from a specific node

Expected results The used resources are displayed
Pass/fail criteria The agents from nodes communicate with master node
Environmental needs TAO platform is up and running
Dependencies
TC-DASH-02: view execution status of jobs
Test item Verifies if the user can visualize the status of jobs (workflow execution)
Covered requirements RQ-PCI-06, RQ-WEM-03
Input specification User input the criteria to see all jobs, or jobs with a specific status
Expected results The status of jobs is displayed
Pass/fail criteria The [Cluster Manager] receive information from each node and communicate with master node
Environmental needs TAO platform is up and running
Dependencies
TC-DASH-03: view job execution details
Test item Verifies if the user can visualize the details of a specific job execution
Covered requirements RQ-PCI-06
Input specification A job is composed by many tasks running on different nodes. User select a specific job to see the status of its tasks
Expected results The details for each job's tasks are displayed
Pass/fail criteria The [Cluster Manager] receive information from each node and communicate with master node
Environmental needs TAO platform is up and running
Dependencies
TC-DASH-04: view platform statistics
Test item Verifies if the Administrator can visualize the platform statistics
Covered requirements RQ-WEM-03
Input specification

The statistics of TAO platform include indicators like:

  • number of successful executed jobs (by user or time period)

  • number of failed executed jobs (by user or time period)

  • amount of used resources (by workflow or time period)

Expected results The platform statistics are displayed
Pass/fail criteria The history collected information into database is coherent
Environmental needs TAO platform is up and running
Dependencies
TC-DASH-05: view user quota status
Test item Verifies if the user can visualize quota status
Covered requirements RQ-DAC-06
Input specification Each user has configured a quota of hard-drive space to be used for input or data product results. The Administrator can see the used space of platform (for all users)
Expected results The user quota is displayed
Pass/fail criteria The [User Quota Management] module is working properly
Environmental needs TAO platform is up and running
Dependencies

Data Visualization#

TC-VISU-01: display data product
Test item Verifies if the platform can display data products (DP)
Covered requirements RQa-UWS-02, RQa-UWS-03
Input specification A workflow is chain of PC who process DP. The result of processing is also a DP which can be pictured into platform interface. The platform is capable to display a limited number of DP types.
Expected results The DP is displayed on platform interface
Pass/fail criteria The [Data Visualization] module is working properly
Environmental needs TAO platform is up and running
Dependencies

Orchestrator#

TC-ORCH-01: cancel a job
Test item Verifies if the user can cancel a running job
Covered requirements RQ-WEM-07
Input specification

From the list with his own jobs, the user select a running job and send the cancel command.

The Administrator can cancel any job.

Expected results The job execution is cancelled and the job status is CANCELLED in Dashboard
Pass/fail criteria The [Cluster Manager] is working properly
Environmental needs TAO platform is up and running
Dependencies TC-DASH-02
TC-ORCH-02: execute a job
Test item Verifies if the user can start execution of a job
Covered requirements RQ-UWS-03, RQ-PCI-04, RQ-PCI-05, RQ-PCI-07, RQ-DAC-01, RQ-DAC-06, RQ-WEM-02, RQa-RMG-08
Input specification

From the list with his own not running jobs, the user select a job and send the start execution command. The input data for the job can be:

  • from local repository

  • from remote repository (SciHub, Amazon WS)

The Administrator can execute any job.

Expected results

During the execution, the job status is RUNNING in Dashboard

At the end of the execution, the job status is COMPLETED in Dashboard

The resulted DP has been transferred into User Catalog

Pass/fail criteria

The user quota allows the job execution

There are eligible nodes where the job's tasks can run

The [Cluster Manager] is working properly

Environmental needs TAO platform is up and running
Dependencies TC-DASH-02
TC-ORCH-03: pause a job
Test item Verifies if the user can pause a running job
Covered requirements RQ-WEM-08
Input specification

From the list with his own running jobs, the user select a job and send the pause command.

The Administrator can pause any job.

Expected results The job status is PAUSED in Dashboard
Pass/fail criteria The [Cluster Manager] is working properly
Environmental needs TAO platform is up and running
Dependencies TC-DASH-02
TC-ORCH-04: resume a job
Test item Verifies if the user can resume a paused job
Covered requirements RQ-WEM-08
Input specification

From the list with his own paused jobs, the user select a job and send the resume command.

The Administrator can resume any job.

Expected results The job status is RUNNING in Dashboard
Pass/fail criteria The [Cluster Manager] is working properly
Environmental needs TAO platform is up and running
Dependencies TC-DASH-02
TC-ORCH-05: distribute job tasks on eligible nodes
Test item Test shows the decomposition of a job into tasks and their distribution into eligible nodes
Covered requirements RQ-UWS-04, RQ-UWS-05
Input specification A WF is converted to a job when is decomposed into tasks. The job is prepared to be executed when its tasks are relocated to eligible nodes.
Expected results

The job's tasks are present into nodes.

The job is ready for execution

Pass/fail criteria

The job's tasks are consistent and recognized by [Cluster Manager]

The [Orchestrator] is working properly

Environmental needs TAO platform is up and running
Dependencies TC-ORCH-06
TC-ORCH-06: relocate job tasks to another execution node
Test item Test shows that is possible to relocate tasks to another eligible node
Covered requirements RQ-RAD-04, RQ-UWS-04
Input specification

The job tasks relocation occurs :

  • during the tasks distribution into eligible nodes

  • when an execution node is flushed

  • when an execution node is deleted

Expected results

During the relocation process the job status is SUSPENDED.

The job is ready to continue the execution.

Pass/fail criteria

The job whose tasks have been relocated is consistent

The [Orchestrator] is working properly

Environmental needs TAO platform is up and running
Dependencies

Scheduler#

TC-SCHE-01: schedule execution of a job
Test item Verifies if the user can schedule the execution of a job
Covered requirements RQa-WEM-01
Input specification After a WF definition, its execution can be scheduled for a specific date
Expected results The WF execution starts at scheduled date
Pass/fail criteria The [Scheduler] is working properly
Environmental needs TAO platform is up and running
Dependencies
TC-SCHE-02: trigger execution of a job
Test item Verifies if the platform can trigger the execution of a job
Covered requirements RQa-WEM-02
Input specification After a WF definition, its execution can be configured to start as result of an event trigger: a new DP is available into User / Resource Catalog.
Expected results The WF execution starts when event is triggered
Pass/fail criteria The [Scheduler] is working properly
Environmental needs TAO platform is up and running
Dependencies

Data Access#

TC-DATA-01a: data connection - use of a local repository
Test item Verifies if the job can use the input DP from a local repository
Covered requirements RQ-DAC-01, RQ-DAC-06
Input specification A workflow is configured with Data Source (DS) pointing to a local repository. The user execute the corresponding job.
Expected results The input DP is copied into input folder of first PC from workflow definition
Pass/fail criteria The connection to local repository failed
Environmental needs TAO platform is up and running
Dependencies TC-ORCH-02
TC-DATA-01b: data connection - use of a remote repository
Test item Verifies if the job can use the input DP from a remote repository
Covered requirements RQ-DAC-01, RQ-DAC-06
Input specification A workflow is configured with Data Source (DS) pointing to a remote repository. The user execute the corresponding job.
Expected results The input DP is transferred into User Catalog, then is copied into input folder of first PC from workflow definition
Pass/fail criteria

The connection to remote repository failed

The user quota is exceeded

Environmental needs TAO platform is up and running
Dependencies TC-ORCH-02

Test procedures#

This section describes the test procedures that will be followed to execute the verification test cases in order to ensure that all requirements are covered.

Test procedure identifier : Title The test procedure identifier followed by a title
Purpose Describe the purpose of procedure
Covered test cases List of test cases (TC) covered by procedure
Preconditions Short description of required context set up before starting the procedure
Steps Describe the steps to be followed in order to achieve the purpose

Table 6‑2 : Test procedure organization

User Authentication#

TP-AUTH-01: user authentication
Purpose TAO platform provide a user authentication mechanism
Covered TCs TC-AUTH-01, TC-AUTH-02
Preconditions N/A
Steps
  • access login form

  • insert valid credentials

User is successfully logged in

TP-AUTH-02: change user password
Purpose User have the possibility to change his account password
Covered TCs TC-AUTH-03a, TC-AUTH-03b
Preconditions User is logged on
Steps
  • ask to change platform account password

  • an email is sent to user account (with a specific token)

  • follow the link from email message and has access to change password form

  • insert a new valid password

Password is successfully changed

User Authorization#

TP-AUTZ-01: access to platform services
Purpose User has access to platform services based on authorization rights
Covered TCs TC-AUTZ-01
Preconditions User is logged on
Steps
  • access functionalities reserved for Administrator

Access is denied

  • access other functionalities

Access is permitted

TP-AUTZ-02: access to User Catalog resources
Purpose User has access to resources only from his own User Catalog (UC), not from other users' catalogs
Covered TCs TC-AUTZ-02
Preconditions User is logged on; UC is populated with resources (DP, WF, PC)
Steps
  • from own UC select a WF

  • from own UC select a DP and attach it as input data for WF

  • save the WF

WF definition is successfully saved

  • select a WF from other UC

Access is denied

TP-AUTZ-03: access to Resource Catalog resources
Purpose User has access to public resources from Resource Catalog
Covered TCs TC-COMP-01
Preconditions User is logged on; RC is populated by Administrator with public resources (DP, WF, PC); RC can contains other resources shared by users
Steps
  • from RC select a WF

  • from RC select a DS and attach it to WF

  • save the WF

WF definition is successfully saved

User Management#

TP-USER-01: user management
Purpose Administrator can manage user accounts
Covered TCs TC-USER-01, TC-USER-02, TC-USER-03, TC-USER-04
Preconditions Administrator is logged on;
Steps
  • create an user

User is displayed in [User Management] module

  • edit user information

  • define user quota

The new information is visible on user account

  • remove user

User account in not visible anymore on [User Management] module

Processing Components Management#

TP-COMP-01a: processing component management - by user
Purpose User can manage own Processing Components from User Catalog
Covered TCs TC-COMP-01
Preconditions Docker Registry node is configured; User quota is defined and large enough;
Steps
  • enter local path to PC archive

  • complete PC descriptor

PC image is created and uploaded to Docker Registry if user's quota allows; PC is available only into UC

  • declare PC as 'public' (share PC for all users)

PC is marked into DB as public; PC is available into RC

  • from UC select the PC and edit parameters; PC is not referenced by any WF

PC description is updated into UC

  • delete the PC from UC; PC is not referenced by any WF

PC is no longer present into UC; PC is un-deployed from Docker Registry

TP-COMP-01b: processing component management - by Administrator
Purpose Administrator can manage Processing Components from Resource Catalog
Covered TCs TC-COMP-01
Preconditions Docker Registry node is configured;
Steps
  • enter local path to PC archive

  • complete PC descriptor

PC image is created and uploaded to Docker Registry if platform quota allows; PC is marked as 'system' and is available into RC

  • from UC select the PC and edit parameters

Access to a PC from users' UC is denied

  • from RC select the PC and edit parameters; PC is not referenced by any WF

PC description is updated into RC

  • delete the PC from UC; PC is not referenced by any WF

PC is no longer present into RC; PC is un-deployed from Docker Registry

User Catalog Management#

TP-UCAT-01: user data product management
Purpose User can manage Data Products from User Catalog
Covered TCs TC-UCAT-01, TC-UCAT-02, TC-VISU-01
Preconditions User quota is defined and large enough; User Workspace (UW) location is defined; Platform temporary storage location is defined;
Steps
  • enter local path to the DP or select a DP from platform temporary location

  • complete DP descriptor

DP is uploaded into UW if user quota allows; DP is available only into UC; DP can be viewed on platform's map-centered graphical interface

  • declare DP as 'public' (share DP for all user)

DP is marked into DB as public; DP is available into RC

  • delete the DP from UC; DP is not referenced by any WF

DP is no longer present into UC; DP is deleted from UW

Topology#

TP-TOPO-01: execution nodes management
Purpose Administrator can manage execution nodes
Covered TCs TC-TOPO-01, TC-TOPO-02, TC-TOPO-03a, TC-TOPO-07
Preconditions Administrator is logged on; Master node is configured;
Steps
  • add a new execution node, noted N1

  • add a new execution node, noted N2

The node is available into RC

  • on physical machine representing the node N1, modify the node's IP and connection credentials

RC report the node as unavailable, the connection with node N1 is broken

  • edit node N1 information introducing the new IP and credentials

RC report node N1 back as available

  • deploy a PC on node N1, then on node N2

  • create a small workflow using the PC above

Workflow tasks are distributed on node N1

  • from RC delete the node N1

The tasks from N1 are moved on eligible node N2; Node N1 is no longer present on RC; The workflow's job is coherent and can be executed

TP-TOPO-02: manage processing components on execution nodes
Purpose Administrator can manage Processing Components on execution nodes
Covered TCs TC-TOPO-03b, TC-TOPO-04, TC-TOPO-05, TC-TOPO-06
Preconditions Administrator is logged on; Nodes are present into RC;
Steps
  • select a node from RC

  • deploy a PC (named PC1) on node (named N1),

The information about node is updated

  • create a workflow using the PC1 and other PCs

  • modify the PC parameters (input/output data types, preconditions for execution, etc.)

The workflow is coherent and can be executed

  • remove, in a coherent manner, the PC1 from workflow

The workflow is coherent and can be executed

  • flush the node N1

All tasks from N1 are relocated to eligible nodes; Node N1 is reported available with no used resources

Workflow Management#

TP-FLOW-01: workflow management
Purpose User can manage his own workflows
Covered TCs

TC-FLOW-01, TC-FLOW-02, TC-FLOW-03, TC-FLOW-04,

TC-COMP-02, TC-DSRC-01

Preconditions There are execution nodes added; The nodes have PCs deployed on them
Steps
  • define a new workflow by adding PC and continuity conditions

  • attach a DS to first PC

The workflow is present into UC

  • clone the workflow

The workflow is duplicated into UC with another name

  • publish the workflow

The workflow is present into RC, but the DS has no connection credentials

  • delete the workflow from UC

The workflow is no longer present into UC (workflow is deactivated)

TP-FLOW-02: workflow coherence
Purpose A workflow must be coherent after modification
Covered TCs TC-FLOW-05a, TC-FLOW-05b, TC-FLOW-05c
Preconditions User is logged on; User have workflows in UC
Steps
  • select a workflow from UC

  • add a PC to workflow between two compatible PCs

The workflow is saved into UC and can be executed

  • for the added PC, modify the execution parameters (only the parameters configured by Administrator to be editable)

The modified parameters are valid; The workflow is saved into UC and can be executed

  • remove the PC from workflow

The workflow is saved into UC and can be executed

Dashboard#

TP-DASH-01: view dashboard statistics
Purpose Platform display statistic information about job execution, used resources, users quota
Covered TCs TC-DASH-01, TC-DASH-02, TC-DASH-03, TC-DASH-04, TC-DASH-05
Preconditions There are jobs with different statuses: RUNNING, COMPLETED, CANCELLED, ERROR, PLANNED. Jobs with status SUSPENDED are visible only during the tasks relocation operation
Steps
  • select to view the "Dashboard" page

  • for his workflows, the user can view :

    • used resources

    • execution status of jobs

    • job execution details (status of jobs' tasks)

  • in addition, the Administrator can view :

    • the platform statistics

    • the quota for all users

Orchestrator#

TP-ORCH-01: job management
Purpose User can manage the execution of his own jobs
Covered TCs

TC-ORCH-01, TC-ORCH-02, TC-ORCH-03, TC-ORCH-04,

TC-TOPO-07, TC-DATA01a, TC-DATA-01b

Preconditions

User has defined a workflow with a DS pointing to:

  • a local repository

  • a remote repository (SciHub, Amazon WS)

Steps
  • select a workflow from UC

  • start workflow execution

The workflow is converted into a job; Job's tasks are distributed into eligible execution nodes; The [Cluster Manager] module start execution of tasks; The job's status into "Dashboard" is RUNNING

  • pause the job

The system waits until the current task is successfully finished and then display the status PAUSED into "Dashboard"

  • resume the job

The job's status is RUNNING

  • cancel the job

The job's status is CANCELED; The last execution node resources are released;

TP-ORCH-02: job distribution on execution nodes
Purpose Job's tasks are correctly distributed on eligible nodes
Covered TCs TC-ORCH-05, TC-ORCH-06
Preconditions Administrator is logged on; There are defined workflows into RC;
Steps
  • from RC, select a workflow with status PALNNED

  • start workflow execution

The workflow is converted into a job; Job's tasks are distributed into eligible execution nodes; The [Cluster Manager] module start execution of tasks; The job's status into "Dashboard" is RUNNING

  • pause the job

The job's status into "Dashboard" is PAUSED

  • from the list of allocated nodes for this job, flush a node

The tasks from flushed node are relocated; The job's status into "Dashboard" is SUSPENDED

  • resume the job execution

The job's status into "Dashboard" is RUNNING

Scheduler#

TP-SCHE-01: execution of a scheduled job
Purpose A job can be scheduled to be executed at a given time
Covered TCs TC-SCHE-01
Preconditions User has defined a workflow into UC; The workflow is scheduled to start at given date; The job status into "Dashboard" is PLANNED
Steps
  • wait until the given date is reached

The [Scheduler] module is triggered to start the job execution; The job's status into "Dashboard" is RUNNING

TP-SCHE-02: trigger execution of a job
Purpose Execution of a job can be started by an event trigger
Covered TCs TC-SCHE-02
Preconditions User has defined a workflow into UC; The workflow has DS defined to use a specific Data Product (named DP1) from RC; The job status into "Dashboard" is PLANNED
Steps
  • wait until DP1 is present into RC

The [Scheduler] module is triggered to start the job execution; The job's status into "Dashboard" is RUNNING