Scripts to collect names for the name-suggestion-index project.
This takes a long time (~1-2h) and a lot of disk space (~75GB). It can be done occasionally by project maintainers.
-
Download the planet
- Mirrors are likely faster than the main repo
curl -L -o planet-latest.osm.pbf https://planet.openstreetmap.org/pbf/planet-latest.osm.pbf
2 choices:
- Make sure your dockermachine has at least 2GB of RAM
- Place the pbf of the planet osm file you wish to process in the same directory as
input-planet.osm.pbf
- Remove the node_modules directory if it exists from a former run (the script will remind you)
- Run
./run.py
- This will also run md5 of the input planet file and write it to last_run.md5 on success.
- That way you can see if it's even worth a bunch of resources to run this script
- Md5 hashes of pbf files are available: https://planet.openstreetmap.org/pbf/
- Install
osmium
command-line tool (may only be available on some environments)-
apt-get install osmium-tool
orbrew install osmium-tool
or similar
-
- Prefilter the planet file to only include named items with keys we are looking for:
osmium tags-filter planet-latest.osm.pbf -R name,brand,operator,network --overwrite -o filtered.osm.pbf
- Run the collection script
- This is complicated because node-osmium is available prebuilt only for older environments. It seems to work ok on Node 10.
node collect_osm.js /path/to/filtered.osm.pbf
git add . && git commit -m 'Collected common names from latest planet'
This project is available under the ISC License. See the LICENSE.md file for more details.