Escaping the Google Photos Ecosystem: A Journey to Self-Hosted Photo Management

In the early days of cloud photo storage, Google Photos (and its predecessor, Picasa) seemed like a perfect solution for backing up our precious memories. With APIs similar to Google Drive, it was easy to use Google Photos as a backup solution. However, as time passed, many of us, myself included, began relying on it as our primary photo storage.

Fast forward to today, and I’ve come to a stark realization: I’m completely locked into the Google Photos ecosystem. The old Picasa APIs are gone, and the current ones come with significant limitations, including stripping EXIF data and resizing images. This realization led me on a journey to export my entire photo library and transition to a self-hosted solution using Immich server.

In this post, I’ll outline the process I used to liberate my photos from Google’s grasp and regain control over my digital memories.

The Limitations of Google Takeout

Google Takeout is often recommended as the go-to method for exporting photos from Google Photos. However, this approach comes with several drawbacks:

1. Metadata Separation: All metadata is extracted and placed into separate JSON sidecar files. These files follow complex naming conventions, making it difficult to pair them with their corresponding media files.
2. Disorganized Structure: Many photos only appear in year folders (e.g., “Photos from 2008”) and need to be manually moved into the correct album folders. This process is complicated by duplicate image names, which get modified during the export.

An Alternative Approach

To circumvent these issues, I developed a hybrid method:

1. Use Google Takeout for Sidecar Files Only: I used Google Takeout solely to obtain the sidecar files containing metadata.
2. Manual Album Downloads: Instead of relying on Takeout for the actual media files, I manually downloaded each album from the Google Photos UI. This resulted in over 500 zip files, which I then extracted.
3. Metadata Reintegration: I wrote custom code to merge the metadata from the sidecar files back into the media files. This step was crucial because any edits made using Google’s UI are only reflected in the sidecar files.

The Process

1. Download and Extract: Manually download all albums as zip files from the Google Photos UI and extract them.
2. Metadata Reintegration: Use custom code to read the sidecar files and add the metadata back to the media files. This includes:
   • Original photo timestamp
   • GPS coordinates
   • Updating the filesystem modified date
3. Handling Edge Cases: Address issues that arise from using the ZIP download method:
   • Replace JPG files with original HEIC files where applicable
   • Correct JPEG file extensions
   • Handle truncated filenames from web downloads

Challenges and Solutions

1. HEIC vs. JPG: The ZIP download sometimes replaces HEIC files with JPG versions. I wrote code to swap these back to the original HEIC format.
2. File Extensions: Some JPG files are downloaded with the JPEG extension. My script corrects these inconsistencies.
3. Truncated Filenames: Long filenames get truncated when downloading from the web. I used the Google Takeout export to recover the original, full-length filenames.

The Code

I’ve made the code I wrote for this project available on GitHub. You can find it here:

• Gist 1
• Gist 2 (Jupyter Notebook)

Feel free to use and modify this code for your own photo liberation project.

Conclusion

Exporting your entire photo library from Google Photos is no small feat, but it’s a worthwhile endeavor if you value control over your data and the flexibility to choose your own photo management solution. By combining manual downloads with custom metadata reintegration, it’s possible to overcome the limitations of Google Takeout and create a more organized, self-hosted photo library.

This process has allowed me to successfully transition to a self-hosted Immich server, giving me full control over my photo collection without sacrificing the metadata and organization I’ve built up over the years.

Remember, while cloud services like Google Photos offer convenience, it’s always important to consider the long-term implications of platform lock-in. By taking control of your data, you ensure that your memories remain accessible and manageable on your own terms.