Description
I have backed up my weaviate collection on the local filesystem of my ‘finland’ weaviate instance. Here is the resulting tree view:
mema@newisa:/sata/backup/20240722060001_mema_wv_backup$ tree
.
├── backup_config.json
└── finland
├── Articles_intfloat_multilingual_e5_large
│ └── chunk-1
└── backup.json
I would now like to restore this collection on another machine, that is hosting another database instance (with the CLUSTER_HOSTNAME set to mema3 and not finland).
Of course I will be able to transfer this backup tree on the machine hosting the new database.
Server Setup Information
Original server:
docker compose based server - definition:
weaviate:
image: cr.weaviate.io/semitechnologies/weaviate:1.25.4
command:
- "--host=0.0.0.0"
- "--port=8080"
- "--scheme=http"
restart: unless-stopped
environment:
LOG_LEVEL: info
ENABLE_MODULES: 'backup-filesystem'
BACKUP_FILESYSTEM_PATH: /tmp/backup
ENABLE_CUDA: 0
LIMIT_RESOURCES: true
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: true
PERSISTENCE_DATA_PATH: /var/lib/weaviate
CLUSTER_HOSTNAME: finland
DISABLE_TELEMETRY: true
GOMAXPROCS: 4
networks:
- weaviate_net
- mema_network
ports:
- "8080:8080"
- "50051:50051"
volumes:
- weaviate_data:/var/lib/weaviate
- /sata/backup:/tmp/backup
logging: *default-logging
destination server - also docker compose based. service definition follows:
weaviate:
image: cr.weaviate.io/semitechnologies/weaviate:1.25.8
command:
- "--host=0.0.0.0"
- "--port=8080"
- "--scheme=http"
restart: unless-stopped
environment:
LOG_LEVEL: info
ENABLE_MODULES: 'backup-filesystem'
BACKUP_FILESYSTEM_PATH: /tmp/backup
ENABLE_CUDA: 0
LIMIT_RESOURCES: true
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: true
PERSISTENCE_DATA_PATH: /var/lib/weaviate
CLUSTER_HOSTNAME: mema3
DISABLE_TELEMETRY: true
GOMAXPROCS: 24
networks:
- weaviate_net
- mema_network
ports:
- "8080:8080"
- "50051:50051"
volumes:
- weaviate_data:/var/lib/weaviate
- /backups:/tmp/backup
logging: *default-logging
you may notice the cluster hostname id is different, the host voume name for the backups is different too, the minor versions differ a bit, the GOMAXPROCS too, the rest is quite identical.
I am trying with the following script on the new “mema3” machine:
#!/bin/bash
curl -X POST -H "Content-Type: application/json" -d '{"id": "20240722060001_mema_wv_backup"}' "http://localhost:8080/v1/backups/filesystem/20240722060001_mema_wv_backup/restore"
after copying the backup directory from the old machine to the following tree on the new mema3 machine:
mema@mema3:/backups$ tree
.
├── 20240722060001_mema_wv_backup
│ ├── backup_config.json
│ └── finland
│ ├── Articles_intfloat_multilingual_e5_large
│ │ └── chunk-1
│ └── backup.json
└── backup_weaviate.sh
So to recap weaviate on the new machine is started as a container with the following docker compose service declaration:
weaviate:
image: cr.weaviate.io/semitechnologies/weaviate:1.25.8
command:
- "--host=0.0.0.0"
- "--port=8080"
- "--scheme=http"
restart: unless-stopped
environment:
LOG_LEVEL: info
ENABLE_MODULES: 'backup-filesystem'
BACKUP_FILESYSTEM_PATH: /tmp/backup
ENABLE_CUDA: 0
LIMIT_RESOURCES: true
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: true
PERSISTENCE_DATA_PATH: /var/lib/weaviate
CLUSTER_HOSTNAME: mema3
DISABLE_TELEMETRY: true
GOMAXPROCS: 24
networks:
- weaviate_net
- mema_network
ports:
- "8080:8080"
- "50051:50051"
volumes:
- weaviate_data:/var/lib/weaviate
- /backups:/tmp/backup
logging: *default-logging
which means the /backups host directory is “mounted” in the container as /tmp/backup
but the curl command fails with a reference to the CLUSTER_HOSTNAME of the old machine (“finland”):
mema@mema3:~/code/aisearch/resources$ curl -X POST -H "Content-Type: application/json" -d '{"id": "20240722060001_mema_wv_backup"}' "http://localhost:8080/v1/backups/filesystem/20240722060001_mema_wv_backup/restore"
{"error":[{"message":"cannot resolve hostname for \"finland\""}]}
how can I fix this? Thanks