Deepsearch Indices

Deepsearch Indices helps user to create an index in AWS Kendra to facilitate deepsearch of the dataset file data.

The following picture depicts the Deepsearch Indices page in Amorphic:

Amorphic Deepsearch Indices contain the following information:

Type	Description
Index Name	Deepsearch Index Name, which uniquely identifies the index.
Description	A brief explanation of the index typically the type of data added to it.
Status	Status of index. Ex: CREATING, ACTIVE etc.
Documents Count	Number of the documents added to the index through datasets.
Auto Terminate	Status of the auto-termination Ex: Enabled, Disabled
Remaining Time	Amount of time (in hr) left for auto-termination
Auto Termination Time	Time at which the system auto terminates the deepsearch index.
CreatedBy	User who created the index.
LastModifiedBy	User who has recently updated the index.

Deepsearch Indices Operations

Amorphic Deepsearch Indices provides below operations for a Deepsearch index:

• Create Index: Create an Deepsearch Index in AWS Kendra.
• View Index: View an existing index.
• Add Datasets: Add datasets to the index.
• Sync Jobs: Sync the documents to the index.
• Search: Search for the indexed document data.
• Delete Index: Delete an existing index.

Create Index

Navigate to side menu and select 'Deepsearch Indices' under Datasets section. You can create a new Deepsearch Index in Amorphic by using the "New Deepsearch Index" (+) button on the right side of the Deepsearch Indices listing page.

In order to create a new Index, you would also require the below information apart from the name and description of the index:

• Auto Terminate: Whether to enable or disable auto termination on the Deepsearch Index. This option enables Deepsearch Index termination to save resource costs based on the termination time value provided by the user. Auto termination process will be triggered every hour and looks for any Deepsearch Indices that needs to be notified or deleted and sends an email when one of the below criteria met. User will receive a notification email in the following scenarios:

  • If the difference between the auto-terminate process trigger run (every whole hour) and the termination time is less than 30 minutes.
  • If the auto-termination process was successfully able to delete the Deepsearch Index after the termination time
  • If the auto-termination process wasn't able to delete the Deepsearch Index due to some fatal errors.

• Auto Termination Time: Denotes the time at which the user wants the Deepsearch Index to be auto terminated. The maximum auto termination time that a user can set will be less than 168 hours (7 days). Once the current time is greater than the termination time then the termination process will be deleting this Deepsearch Index at the next whole hour.

  Note

  • Auto-termination process is scheduled to run every hour on the hour (e.g: 6:00, 7:00, 8:00, 9:00).
  • User will receive a email notification only when the user is subscribed to alerts. Please refer to Alert Preferences to enable alerts.
  • When the termination time elapses, auto termination process will terminate/delete the Deepsearch Index and also deletes all the metadata related to the Deepsearch Index and this process cannot be undone.

Once the index is created with the above metadata, index will be in CREATING state. It'll complete creating the index in few minutes and status will be ACTIVE.

Add Datasets

User can add datasets to the index using the 'Add datasets to Index' (+) button in the Datasets tab. Note that only datasets with unstructured files can be indexed. Refer AWS Documentation for more information on type of supported documents.

Please follow the below animation on how to add datasets to the index:

Sync Jobs

Once the datasets are added to the index, User can run 'Sync job' to add the documents (unstructured and supported) from the datasets to the index. It can be done using the 'Run sync-job' button in the 'Sync-Jobs' tab.

Once the sync job is triggered and job will be in SYNCING state. User can do following actions when the job is in SYNCING state:

1. User can get the latest status of the job by refreshing the jobs using 'Refresh' button on the right side.
2. User can stop the syncing of the datasets using the 'Stop' (red) button beside the SYNCING status of the respective job.

Also, user can view the details of the job like Status, Datasets synced, Error message (if failed) etc.

Please follow the below animation on how to run the sync-job:

Search

After running the sync-job and once documents are indexed, User can search for data in the documents using the search bar in the 'Search' tab.

Upon the keyword search, it will be searched through all the indexed documents and results will be displayed with following options:

1. Add/Remove tags: User can add tags to the files and remove tags from the files.
2. Download: User can download the file.
3. File Preview: User can preview the file. Supported file formats are txt, json, docx, pdf and html.
4. View Dataset: User can navigate to the respective dataset in which the file resides.

Please follow the below animation on how to run the sync-job:

View Index

If the user has sufficient permissions to view an index then all the index information can be viewed by clicking on the Index name in the “Deepsearch Indices” under Datasets section.

Following details will be displayed in the index details page:

Following details will be displayed when the user enables auto-termination on the deepsearch index. Remaining Time denotes the amount of time (rounded to nearest upper hour) left for auto-termination.

In below image, the auto termination time is set to 02 Jun, 2021 6:51 PM but the deepsearch index will be deleted at 02 Jun, 2021 7:00 PM because the termination process is scheduled to run at whole hour.

In the details page, Estimated Cost of the deepsearch index is also displayed to show approximate cost incurred since the creation time.

Delete Index

If the user has sufficient permissions to delete an index then the index can be deleted using the Delete (trash) button on the right side.

Please follow the below animation to delete the index: