opendata-koop

ArcGIS Open Data Koop Application

The Koop application that serves downloads in opendata.arcgis.com.

Setup

$ git clone [e-mail]:ArcGIS/opendata-koop.git
$ cd opendata-koop
$ cp config/default.example config/default.json

Management

Scripts

• node scripts/drop-failed.js: removes all resources with status: 'Failed'
• node scripts/drop-processing.js: removes all resources with status: 'Processing' that are older than 24 hrs
• [DEPRECATED] node scripts/drop-processing-legacy.js: removes all resources with status: 'Processing' that are from Koop-Agol 1.x
• [DEPRECATED] node scripts/inspect-failed.js: logs the ids of all datasets with `status in ('Failed', 'failed', 'Processing')

API

• get /agol/queue/length: gets number of agol import jobs waiting for processing
• get /agol/queue/working: get the count of import jobs currently processing
• get /agol/queue/workers: gets the status of all agol import workers
• get agol/queue/clearFailed: remove all failed jobs from the queue and drop the resources from the database
• Note: this is run by the app server automatically every 30 minutes

Docker Examples

The following is only supported on Linux hosts.

$ docker build -t opendatakoop .
$ docker run -it --rm -v $PROJECT_ROOT/config:/usr/src/koop/config -v /var/log/koop:/usr/src/koop/log opendatakoop /bin/bash