I recently have to migrate Rutgers Plus from Firebase to my self-hosted mongodb (Atlas) because of the Chinese great firewall.<\/p>\n
First install gsutil<\/a>, firebase-tools<\/a> and mongo shell<\/a>.<\/p>\n To migrate data collections one at a time, create and run a bash file:<\/p>\n where $1 will be the name of the collection. In the file, I converted the id from firebase’s “id as the key” format to mongodb export’s “id as the value” format. Also mongoexport doesn’t strictly follow the JSON standard as it requires an array without comma separators instead of an object with id-as-keys. I did the tranformation as well.<\/p>\n To download static files from Firebase for re-uploads:<\/p>\n To migrate users, find password hash parameters in the Firebase authentication tab, you get {key} {salt separater} {rounds} {memcost} there. Download the users info csv:<\/p>\n and mark the column right before the first name and last names of the users. Those are {salt base} for each user.<\/p>\n We need the same algorithm firebase used for hashing passwords to keep using the existing password hashes, or we need to ask the users to reset their passwords or authenticate with firebase in the future to migrate users one at a time. These two methods are also the official methods from AWS sources.<\/p>\n Luckily after some decent researching, I learnt that Firebase kindly provides their modified SCRYPT hashing algorithm on Github. Therefore, on the server:<\/p>\n You can use this tool for password hashing in the future.<\/p>\n At last we uploads the authentication info up to the new database. I cut out the thrid party provider fields because I’m not using them.<\/p>\n I recently have to migrate Rutgers Plus from Firebase to my self-hosted mongodb (Atlas) because of the Chinese great firewall. First install gsutil, firebase-tools and mongo shell. To migrate data collections one at a time, create and run a bash … Read More<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false},"categories":[7,6],"tags":[],"jetpack_featured_media_url":"","jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p6eWPT-3q","_links":{"self":[{"href":"https:\/\/zhouanbo.com\/index.php?rest_route=\/wp\/v2\/posts\/212"}],"collection":[{"href":"https:\/\/zhouanbo.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zhouanbo.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zhouanbo.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/zhouanbo.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=212"}],"version-history":[{"count":11,"href":"https:\/\/zhouanbo.com\/index.php?rest_route=\/wp\/v2\/posts\/212\/revisions"}],"predecessor-version":[{"id":233,"href":"https:\/\/zhouanbo.com\/index.php?rest_route=\/wp\/v2\/posts\/212\/revisions\/233"}],"wp:attachment":[{"href":"https:\/\/zhouanbo.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=212"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zhouanbo.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=212"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zhouanbo.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=212"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}echo \"Exporting from firebase..\"\nfirebase database:get \/$1 --project <PROJECT_NAME> --pretty > $1.json\necho \"Switching id...\"\nawk '$0 ~ \/^ \"\/ {split($0, a, \"\\\"\"); print \"\\{ \" \"\\\"_id\\\": \" \" \\\"\" a[2] \"\\\",\"; next}{print}' $1.json > $1_tmp.json\necho \"Converting to array...\"\nawk 'NR==1{print \"[\";next}\/^}\/{print \"]\";next}{print}' $1_tmp.json > $1_array.json\necho \"Uploading to mongodb...\"\n$HOME\/...\/mongodb-osx-x86_64-4.0.0\/bin\/mongoimport --host <MONGO_HOSTNAME> --ssl --username <USERNAME> --password <PASSWORD> --db rutgersplus --collection $1 --jsonArray --file $1_array.json\n<\/code><\/pre>\ngsutil cp -r gs:\/\/<PROJECT_NAME>.appspot.com\/users .\n<\/code><\/pre>\nfirebase auth:export --project <PROJECT_NAME> auth.csv\n<\/code><\/pre>\napt-get install autoconf\napt-get install build-essential\napt-get install openssl\napt-get install libssl-dev\ngit clone https:\/\/github.com\/firebase\/scrypt.git\ncd scrypt\nautoreconf -i\n.\/configure\nmake\necho `.\/scrypt {key} {salt base} {salt separator} {rounds} {memcost} -P <<< \"password\"`\n<\/code><\/pre>\ncut -d ',' -f 1-7,25-26 auth.csv > auth_cut.csv\n\necho \"_id,email,isVerified,passwordHash,passwordSalt,name,photoUrl,createdAt,signedInAt\" | cat - auth_cut.csv > auth_header.csv\n\n$HOME\/...\/mongodb-osx-x86_64-4.0.0\/bin\/mongoimport --host <MONGO_HOSTNAME> --ssl --username <USERNAME> --password <PASSWORD> --db rutgersplus --collection auth --type csv --headerline --file auth_header.csv\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"