Skip to main content

Brainspace

Brainspace Implementation Guide

Latest Release Announcement

As of September 30, 2022, our latest release is version 6.7 and its subsequent point releases. See our published Release NotesRelease Notes for the latest details on features and bug fixes. This guide is designed to help our clients quickly and easily deploy or upgrade the Brainspace application. If you have any questions, please contact Brainspace Support at support@revealdata.com for assistance.

Brainspace_6_Client_Browser_support.png
Brainspace_6_AppSrv_OnDemandAnalytics_update.png
Brainspace_6_AnalyticsSrv_Connectors_update.png

Brainspace 6.7 Server Guidelines

Server Resource recommendations vary depending on client data, document and dataset size, complexity and the Brainspace features being used.

Use these server recommendations as a guideline only: monitor system resource utilization closely and update as required.

If using VM instances, use Resource Allocation Reservations to ensure committed server performance.

Brainspace_6_recommendation_specs.png

For larger volume instances, please consult your Brainspace Solutions Architect or submit a request to support@revealdata.com.

Additional Sizing Considerations

  • The provided Brainspace 6.7 Server Guidelines will provide for full Brainspace functionality and performance for a wide range of document populations. Actual resource needs will vary depending on source documents complexity and the combination of Brainspace analytics being used.

  • Brainspace processes all extracted text and metadata without a requirement for pre-filtering, however large, complex data types can reduce system performance and require additional resources.

  • Larger datasets tend to consume more RAM than multiple smaller datasets due to larger brains and more complex relationships within the document population.

  • Server specific resource considerations.

    • Application Server – All Active datasets are loaded into RAM for optimal graphical response. Large documents and datasets will increase memory requirements. Memory and Processor utilization is driven by Concept Search, Cluster Wheel and Communications Analysis activity.

    • Analytics Server – Only one dataset can be built at a time on the Analytics server. Additional Analytics servers can be deployed to allow for multiple, parallel dataset builds to occur. Additional resources (RAM & Processors, always at least a 4:1 ratio) may be required for very large dataset builds or may help improve dataset build times.

    • On-Demand Analytics (ODA) Server – Only one Focus can be built at a time on the ODA server. Additional ODA servers can be deployed to allow for multiple, concurrent Focus builds. Also, machine learning activity for CMML and PC occur on the ODA server and heavy classifier users should increase system resources accordingly. Additional resources (RAM & Processors, always at least a 4:1 ratio) may be required for very large Focus builds or may help improve CMML, PC and Focus build times.

  • Increased user activity in Brainspace does not drive system resources quite as much as data and system activity. Plan accordingly for dataset volume (number and size of documents) and active projects concurrently in the system. A good rule of thumb for high user counts is to add 0.5GB per active user (i.e. 20GB for 40 concurrently logged in users) to the Application Server.

  • Disk usage on the /data volume will expand by up to six times the size of your source text during the Brainspace dataset build process, index creation and optimization. Following the dataset completion and cleanup process the data will typically occupy from 1-2x the source text on disk.

  • Keep in mind that usage requirements will likely change between minor versions (6.5 to 6.6, 6.6 to 6.7, etc.) as new features are added.

Disk Layout

Brainspace recommends 100GB of disk storage for each server, utilizing the following layout. Please note that any deviations from this recommendation may impact upgrades or application performance and should be discussed with the Brainspace Support PM or Engineer prior to upgrade or install. Any file storage for client-managed logging or security applications should be in addition to this recommendation.

The /localdata volume required on the Application and On-Demand Analytics servers requires a high-performance storage solution to process large volumes of data for the Brainspace machine learning and advanced graphical analytics features. SSD disks or a High IOPS (≥ 30,000) Enterprise storage solution is recommended, with 10Gbps storage network.

The /data volume must be shared from the Analytics server to all other servers. Brainspace recommends using NFS to share this volume from the Analytics server to the other severs in the solution, however it may also be shared from an Enterprise storage solution with minimum 10,000 IOPS and at least 1Gbps actual throughput between servers.

The /localdata-share (/localdata on App) is the NFS share from the Application Server to the ODA Server that was a new requirement introduced for 6.6. The NFS must be configured before install, just like /data. During the actual install, the installer asks where the localdata share is, which should be /localdata-share on the On-Demand Server. This is in addition to /localdata and not to replace /localdata.

Information on configuring the /localdata-share can be found here:

This is our recommended disk layout. Alternatives can be configured if desired, so long as the general layout remains as follows:

Mount Point

Volume Size

File System

Application Server

/boot

250 M

Ext4

/var

50G (' 25G free for upgrade)

Ext4

/tmp

20G

Ext4

/

Remainder

Ext4

SWAP

8GB maximum

SWAP

/localdata (NFS shared to the ODA server)

Variable

XFS

/data (NFS shared from Analytics server)

Variable

XFS

Analytics Server

/boot

250 M

Ext4

/var

50G (' 25G free for upgrade)

Ext4

/tmp

20G

Ext4

/

Remainder

Ext4

SWAP

1x Server RAM

SWAP

/data (NFS shared to App & ODA servers)

Variable

XFS

On Demand Server

/boot

250 M

Ext4

/var

50G (' 25G free for upgrade)

Ext4

/tmp

20G

Ext4

/

Remainder

Ext4

SWAP

1x Server RAM

SWAP

/localdata

Variable

XFS

/localdata-share (NFS shared from App server)

NFS

/data (NFS shared from Analytics server)

Variable

XFS

OS Configuration Requirements
  • Brainspace 6.7 requires either CentOS or Red Hat Operating System, Version 7.6 with current patches thru 7.9.

  • The US English version of CentOS or Red Hat should be selected. Other languages may cause data format issues.

  • Use the ‘Minimal Install’ option to prevent installation of unnecessary packages.

  • Perform a YUM update to update installed packages to the latest available security and application fixes.

  • Install the following Linux utilities to your servers for use during installation or subsequent troubleshooting. Your servers will require internet access, or access to a local RPM repository, to install the appropriate packages.

    • Execute the following command to install the recommended packages:

      yum -y install nfs-utils lvm2 sysstat nmap

  • If your corporate security policy requires other Linux configuration settings, please notify your Brainspace Project Manager, as alternate settings may prevent Brainspace from installing or functioning as required.

  • Starting in Brainspace 6.6 the perl-libs package has been required. In order to install this Perl will need to be installed. This can be installed with the command:

    yum -y install perl-libs

Shared Storage Configuration

Brainspace requires a shared file system for the /data volume available to all Brainspace servers. We recommend using a Network File System (NFS) with nolock option enabled to share the /data storage volume hosted from an enterprise network storage (NAS) or from the Analytics server to the other servers in the instance.

For 6.7, we also require that the /localdata/brainspace on the App server be shared to the ODA Server as /localdata-share.

NFS Configuration Summary /data

Analytics Server Export Path: /data

Application Server Mount Path: /data

On Demand Analytics Server Mount Path: /data

NFS Configuration Summary /localdata-share

Application Server Export Path: /localdata/brainspace

On Demand Analytics Server Mount Path: /localdata-share

Brainspace Network Summary
Brainspace_6_Network_Summary.png

Important

If using a load balancer or proxy service to access Brainspace, it is recommended that the load balancer be configured to bypass HAProxy, handle any required SSL processing and additional security requirements and connect to the Brainspace Application server on port 8081.

See our HAProxy usage article for more information.

The haproxy service name changed from 6.1 to 6.2 and 6.7 to brainspace-haproxy. Here's a summary of the configuration and changes: Replacing the Default Brainspace Self-Signed SSL Certificates.

Firewall Access Rules Summary

See Appendix B for table of port allowances.

Security Settings

Linux System Accounts

  • Secure and store your ‘root’ account password in a safe place. Brainspace will not store nor be able to recover this account if lost.

  • Additional individual accounts should be created for other administrators within your organization, as necessary, to be used for server login, troubleshooting, file copy, etc.

  • Disable root login (recommended for added security) once you have created the suggested individual login accounts.

  • The ‘brains’ service account will be created by the Brainspace installer. Do not set a password for this account. You may sudo as the brains user as necessary for troubleshooting.

Brainspace Accounts

A ‘support@revealdata.com’ Super Admin account will be created for your initial Brainspace user interface (UI) access. The Super Admin has the highest level of permissions for completing the installation configuration requirements, as well as for ongoing administrative and troubleshooting tasks across all Groups and Datasets. Brainspace recommends creating individual User, Group Admin and Super Admin accounts for individuals on your team as required to perform their assigned tasks. The original Super Admin account, or other individual account with system-level permissions, will be needed by Brainspace Support personnel should we need to assist your team for future troubleshooting issues. Please do not change the password, or if you do, keep the password in a safe and secure location.

Note

If your corporate security policy conflicts with any of these recommendations, your security policy will take precedent. Please discuss any concerns with your Project Manager to avoid disruption to your application functionality.

SSL Certificates

By default, all Brainspace interfaces (UI and inter-server communications) use self-signed SSL

certificates. You may provide your own domain signed certificate for application access and encryption, if required.

Provide certificate in PEM (Base64-encoded ASCII) format, an unencrypted private key, and any intermediate certificates.

Brainspace can also be configured to offload SSL-encryption to a load balancer or proxy server. Please let your Project Manager or Solutions Architect know if you plan to do SSL encryption on another device.

Scheduling Brainspace 6.7 Installation or Upgrade

Planning

Once your servers are built and the environment configured as required, perform an audit of your systems to verify all prerequisites are completed to avoid delays during, or risk rescheduling, the installation.

Brainspace provides an automated audit script to help audit and verify your servers are ready for the installation or upgrade. The script gathers relevant server specifications (processors, memory, swap, etc.) in a text file for easy review or to submit to Brainspace for review prior to scheduling installation resources. See our Infrastructure Audit article for full audit details: Preinstallation Script: Server Configuration Audit.

Resolve any exceptions noted in the audit results. Brainspace requires a clean system audit before installation resources can be scheduled. If you have questions or are unable to complete any of the configuration requirements on your own, please contact your Project Manager to discuss your questions or to schedule a technical review call with one of our engineers to address any concerns regarding environment build and configuration.

Installation or Upgrade

Brainspace installation resources normally won’t have access to your servers and will rely on a screen sharing application ie; WebEx, Zoom, etc. The screenshare presenter must have full access to the server environment where Brainspace will be installed.

Brainspace application installation or upgrade generally take between 2-4 hours, depending on server access and applicable integration with your third-party applications.

Key Resources Required for Installation

IT (server, storage and network) – Available to assist with server access and to remediate any environment issues that may hinder the Brainspace installation/upgrade process.

Relativity®/Nuix Admins (as applicable) – Access accounts, plus the ability to access test data within Relativity®/Nuix, will be required during the installation. Please make sure you have a test workspace in Relativity®, or a test case in Nuix, to verify functionality after installation.

Brainspace Users/Admins – Following the installation or upgrade, we will perform a brief product orientation, including demonstration on data ingestion and the key application features, to prepare your team to begin immediately realizing the power and new features in Brainspace 6.7. We will also show how to troubleshoot common issues, download relevant logs and submit questions/problems to Brainspace Support when you need assistance. All Brainspace Users and Admins are encouraged to participate.

Brainspace Integration Requirements

Relativity® Integration for versions 9.7+ and 10.x+ (including RelativityOne cloud)

Brainspace has included a new connector as of version 6.2 with improved security, performance and functionality over our legacy connector. The new Relativity® Plus connector requires only Relativity® API access via port 443 (https) and only works with Relativity® versions 9.7 and newer. In Brainspace 6.7 you can use the Relativity® Connector on datasets originally built with the Relativity® Connector, as long as the “Use Legacy Names” toggle is on.

Relativity® Integration for versions 9.2 – 9.7 (on-prem deployments only)

Brainspace integrates with older versions of Relativity® using our legacy Relativity® connector. This connector requires access to the Relativity® REST API (https) and direct SQL access (tcp 1433) using Brainspace legacy Relativity® connector.

Brainspace’s legacy Relativity® connector uses the RSAPI service to determine user access permissions to assigned workspaces, and MS-SQL access for high-speed ingestion and sync of Brainspace data back to Relativity®. The Brainspace Relativity® plugin respects the Relativity® ACLs (access control list) for data streaming and overlay back to Relativity® – Workspace access and updates will only occur with proper user authentication credentials.

Integrating with a Single Relativity® SQL Instance

  • Brainspace utilizes TCP port 1433 (by default; please indicate if you are using a non-standard SQL port) to access the Relativity® MS-SQL database for integration.

Integrating with Multiple/Distributed Relativity® SQL Instances

  • If using Static ports – Configure each SQL instance with the same port (typically 1433).

  • If using Dynamic ports – The MS SQL Browser Service must be configured (typically port 1434 UDP) for Brainspace to determine the actual port in use for each SQL instance. The full range of dynamic ports in use must be allowed through the firewall from the Brainspace Application server to each Relativity® SQL instance.

The Relativity® MS-SQL instance must be configured and available prior to Brainspace installation. This account must have EDDSDBO user rights. If unable to use EDDSDBO account, the provided account permissions will need to either be added manually or a script written to automatically update the “brainspace” user to all new workspaces.

  • Server Role = public

  • For Workspaces to be used with Brainspace, the database user needs db_datareader, db_datawriter, db_ddladmin and roles

  • For the main EDDS database, the database user needs db_datareader. For enhanced security, a custom role providing read only access to the Case and ResourceServer tables may be used.

  • For additional security, set the minimum required column access in Relativity® EDDS db:

    • Case ArtifactId

    • Case ServerId

    • ResourceServer ArtifactId

    • ResourceServer Name

      • The MS-SQL account must be configured on all SQL instances in a distributed SQL environment.

Nuix Integration

  • Nuix integration requires the Nuix RESTful Service to be installed and configured on your Nuix Server. Go to Nuix.com to see references for common Nuix versions.

Verify the Nuix RESTful API service is configured and listening by executing a curl command to invoke the Nuix health check from the Brainspace Application (Runtime) server.

If NUIX RESTful API interface is configured for SSL:

curl -k https://nuix servername:8443/nuix-restful-service/svc/v1/system/health

If NUIX RESTful API interface is configured for non-SSL:

curl http:// nuix servername:8080/nuix-restful-service/svc/v1/system/health

Expected Result: "success" : true

Appendix A: Pre-Install Checklist: Server Infrastructure Readiness

Fill out the Pre-Install Checklist to ensure a successful Brainspace installation. Completion is required before installation resources can be scheduled. Contact your Project Manager if you need assistance or have questions with any configuration item. (double-click the checkbox to mark item completed)

Check

Items to Complete

Additional Information

Server Configuration

Confirm server configuration (OS, Cores, RAM, Disks, Swap, DNS, NTP, patching, etc.) meets Brainspace required specifications.

Run automated system audit script and resolve any identified issues.

Full Audit instructions

Upload final audit script results to support@revealdata.com.

  • Check to indicate all items complete.

SSL Certificate

Which certificate type will you use?

Select all that apply:

  • Default self-signed

  • Customer-provided certificate

  • Other. Please specify:

Install and License Files available

Copy Brainspace installation files (installer + license) to /data directory on Application server and to root (/) on the On-Demand Analytics server.

  • Files copied to Application and On-Demand Analytics server

Reboot servers

Reboot all servers to ensure services startup properly before beginning the Brainspace installation.

Restart the Analytics Server, the Application Server, then the On-Demand Analytics Server to ensure NFS services come online.

0 Servers rebooted

(After NFS configuration & Before Brainspace installation)

Analyst Readiness

Have all of your users (Analysts and Administrators) received training on Brainspace 6?

0 Yes, our users are ready!

0 Please send me more information on training.

Brainspace offers both self-guided and formalized training opportunities.

Self-Paced

https://help.revealdata.com – Updated User Guides

Brainwaves – Online Admin and Analyst training

Environment Security Standards

FIPS Compliance

0 Compliance restrictions discussed with PM

Supported Browsers

Confirm your users have Chrome, Edge (Chromium-based versions) or Firefox installed with WebGL enabled

0 Users have a supported browser

Appendix B: Pre-Install Checklist: Network Access

Source

Destination

Ports

Description

Brainspace Common Ports

End Users

Application Server

443 (HTTPS)

User access to User Interface

System Admin

Application, Analytics, ODA Servers

22 (SSH)

Admin access to Secure Shell for System Administrative tasks

Analytics & ODA Servers

Application Server

8081

(HTTPS/Tomcat)

Inter-server communication to the Application Server

Application Server

Analytics & ODA Servers

1604 (HTTPS)

Inter-server communication from the Application Server

Application Server

Analytics Server

111 TCP/UDP (RPC) 2049 TCP/UDP (NFS)

20045 TCP/UDP (STATD)

20047 TCP/UDP (LOCKD)

20048 TCP/UDP (MOUNTD)

Network file sharing for /data

ODA Server

Application Server

111 TCP/UDP (RPC) 2049 TCP/UDP (NFS)

20045 TCP/UDP (STATD)

20047 TCP/UDP (LOCKD)

20048 TCP/UDP (MOUNTD)

Network file sharing for Application’s /localdata/brainspace to ODA’s /localdata-share

Application, Analytics, ODA Servers

Application Server, or

PostgreSQL Server if combined

5432 (PostgreSQL)

Communications to the PostgreSQL database

Relativity® Integration Ports

Application Server

Relativity® RSAPI Server

443 (HTTPS)

Relativity® REST API access

Application Server

Relativity® MS SQL Server(s)

1433 TCP (SQL) 1434 UDP (SQL Browser Service)

Relativity® SQL Access, only if using Relativity® Classic connector

Relativity® Integration Ports

Application Server

Nuix RESTful API Server

8080 (HTTP) or 8443 (HTTPS)

Nuix RESTful API access

Appendix C: Post-Install Checklist (Brainspace Verification)

Complete this checklist after installation to ensure Brainspace is ready for use by your team.

Check

Additional Information

Results

Verify Admin functions

Datasets: Add from DAT file (requires Connector) and view dataset info.

Create tags. Download Archive, Process, Status reports; Build Error and Build logs and Import Error Archive. Explain each.

Users and Groups: Create ‘customer-super@email.com ()’ Super Admin

account. Create test Group, User and Notebook; grant access to test Notebook. Show how User cannot access other Group Notebooks.

Connectors: Setup Relativity® or Nuix, DAT connectors

Services: show license status, documents in use, expiration date and

disk and memory usage. Download brainspace.log & catalina.out logs from App, analysisServer.out and analysisserver.log files from Analytics servers. Upload to install ticket.

Errors: View latest. Archive.

Create Enron10k dataset from local disk

Verify build completes to ensure functionality. Performance wise, we expect a full build to complete in 6 to 7 minutes.

Build Enron10k from DAT

Perform build from DAT completes without errors

Concept Search & Cluster Wheel

Search for ‘birthday’ and verify results on Cluster Wheel. Top Terms (time, day, good, work, mom, great, etc.) and Top Concepts (happy birthday, birthday party, celebrate, cake, tortillas, dinosaur, annie, brisket, etc.). Add block of Terms from Cluster Wheel, verify further filtering on Dashboard.

Create a Notebook

Create a Notebook from search results.

Add to a Notebook

Remove Duplicate documents. Add search results to prior saved Notebook.

Create a Focus

Remove block of Terms filter, use results to Create Focus.

Communication Analysis

Select 2 people while holding Shift key. Verify people are highlighted and results display for Sent (unique docs, To, cc, bcc, etc.), Received, Sends Most To, Receives Most From, etc.

Near Duplicates

Verify ND documents at Cluster Wheel edge. Add to search criteria.

License Verification & Usage Reporting

Verify license usage and expiration details (Admin\Services\Application).

Download and send to Brainspace (reports@revealdata.com)

Remove Unused Plug-Ins (Brainspace Only)

Navigate to /var/lib/brains/.brainspace/plugins and remove applicable, unused plug-ins

Date Format

Show AUTO (US ISO) and EURO Date Format plus the ability to custom configure any consistent date format.

Capture Version number

Access the server details under the Services tab

Appendix D: Post-Install Checklist (Relativity® / Nuix)

We will complete this checklist after installation to ensure Brainspace is ready for use by your team.

Relativity® Checks

(if applicable)

Description

Additional Information

Results

1

Relativity® applications

Verify install

Confirm the Brainspace Addons Application is installed to at least one workspace.

2

Relativity® Connector Set Up

Add database details (URL, ports, password, etc) and Fields

Confirm after setting up connector. Select ALL overlay fields.

3

Dataset

Create Dataset via Ingest Wizard

Apply customer supplied credentials, create the following Field mappings: *DOCID, *Fulltext (bodytext), Date, DateSent, To, From, CC, BCC. Facet To, From, CC, Date and BCC.

(*Minimum required fields)

4

Dataset

Submit Dataset

During submission, tail brainspace.log and info.log to watch for any errors

5

Sync

Sync Notebook

Confirm functionality via results returned (e.g. has images)

6

Sync

Relativity® Overlay (Test in Relativity®)

Automatic overlay test via executing ‘BD all coding fields’ saved search. Confirm EMT fields populated.

7

Validate

Record Count

# of records and time for build

8

Sync

Application Library

Push Brainspace application from Test workspace to Relativity® Application Library and unlock (push to library)

Nuix checks (if applicable)

1

Dataset

Create Dataset

Create Dataset from Nuix test case

2

Export

Initiate Export

Verify Export

3

Sync

Test sync

Tag documents and sync to Nuix

Appendix E: Start-Up Order for Servers and Services

Use the following guidelines for server and services start-up:

  1. If you are using NFS to share the /data volume from the Analytics server to other servers in your environment, the NFS service on the Analytics server must be started before NFS client instances. This typically means you would start your Analytics server first, and all required services should start automatically (especially NFS, analysis server).

  2. Start the Postgres database next if Postgres is deployed on a dedicated server (not on the Application server).

  3. If Postgres is installed on the Application server, start the Application server, and all required services should start automatically (especially Postgres, NFS client, and Tomcat).

  4. Start the On-Demand Analytics server next, and all required services should start automatically (especially NFS and analysis server).

  5. If you have multiple, redundant Analytics or On-Demand Analytics servers, those should be started at this point.