How to migrate NSPersistentCloudKitContainer to App Groups

Recently I needed to add iOS Widgets to my app Wins. As I discovered, if you are using Core Data and want to share data between your iOS app and widgets, you need to use App Groups.

At first, I tried to migrate my database following the solution by Donny Wals. In his book Practical Core Data, there is a separate chapter dedicated to this topic: “Chapter 6 - Sharing a Core Data store with apps and extensions”. If you work with Core Data, I highly recommend reading this chapter and the whole book. Donny is a great Core Data expert and his book helped me a lot with understanding this framework.

However, in his example, he is using NSPersistentContainer. And in my app, I am using NSPersistentCloudKitContainer, which means that I have Core Data with CloudKit support. When trying to migrate my database I ran into 2 problems:

My app was freezing during migration if iCloud was turned off (for example, if the user was not signed in to iCloud).
After migration, all my data was duplicated.

I spent many hours until I solved both issues. Using ChatGPT, reading Apple's documentation, and browsing through StackOverflow and Apple Developer Forums. As there wasn't one place with a complete working solution, I decided to write this article as it might help other iOS developers.

At the end of this article, I will show you my entire PersistenceController code. I want you to understand each part of the code, so let's have a look at each part of my PersistenceController. But first, a few words about App Groups.

App Groups

What are they and why do we need them? By default, when an app uses Core Data, the app's database is created in the Application Support directory of this app. And only this particular app has access to this folder (and database). App Groups are shared folders. When 2 or more apps have access to this folder, they all can use the same database.

You can read how to configure App Groups in this article from Apple:
https://developer.apple.com/documentation/xcode/configuring-app-groups

Some people recommend adding App Groups support to all new projects as doing this at the beginning is much easier than migrating an existing database to the App Group folder.

In this article, I will show you how I migrated my database to the App Groups folder. First, I will show you my whole code for the PersistenceController. After this, I will talk about each part and explain why I did something the way I did.

Full code

import CoreData

struct PersistenceController {
    static let shared = PersistenceController()

    // Create a database for Preview Canvas.
    static var preview: PersistenceController = {
        let result = PersistenceController(inMemory: true)
        let viewContext = result.container.viewContext

        for _ in 0..<4 {
            let newGoal = CDEGoal(context: viewContext)
            newGoal.type = GoalType.number.rawValue
            newGoal.date = Date()
            newGoal.icon = "trophy"
            newGoal.name = "Goal name"
            newGoal.goal = 1000
            newGoal.currently = 500
            newGoal.status = GoalStatus.planned.rawValue
            newGoal.color = "purple"
            newGoal.id = UUID()
        }

        do {
            try viewContext.save()
        } catch {
            let nsError = error as NSError
            fatalError("Unresolved error \(nsError), \(nsError.userInfo)")
        }
        return result
    }()

    let container: NSPersistentCloudKitContainer

    var oldStoreURL: URL {
        let appSupport = FileManager.default.urls(
            for: .applicationSupportDirectory,
            in: .userDomainMask
        ).first!
        return appSupport.appendingPathComponent("YourAppName.sqlite")
    }

    // We define new App Group containerURL.
    var sharedStoreURL: URL {
        let id = "group.com.yourDomain.YourAppName" // Use App Group's id here.
        let containerURL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: id)!
        return containerURL.appendingPathComponent("YourAppName.sqlite")
    }

    init(inMemory: Bool = false) {
        container = NSPersistentCloudKitContainer(name: "iCloud.com.yourDomain.YourAppName")

        let description = container.persistentStoreDescriptions.first!
        let originalCloudKitOptions = description.cloudKitContainerOptions

        // Use the App Group store if migration is not needed (if default store without App Group doesn't exist).
        if !FileManager.default.fileExists(atPath: oldStoreURL.path) {
            description.url = sharedStoreURL
        } else {
            // Disable CloudKit integration if migration is needed.
            description.cloudKitContainerOptions = nil
        }

        description.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey)

        if inMemory {
            description.url = URL(fileURLWithPath: "/dev/null")
        }

        // Load persistent stores.
        container.loadPersistentStores(completionHandler: { (storeDescription, error) in
            if let error = error as NSError? {
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        })

        // Perform the migration.
        migrateStore(for: container, originalCloudKitOptions: originalCloudKitOptions)

        container.viewContext.automaticallyMergesChangesFromParent = true
    }

// Function migrateStore. Migrates data store to App Group if needed.
func migrateStore(for container: NSPersistentCloudKitContainer, originalCloudKitOptions: NSPersistentCloudKitContainerOptions?) {
    let coordinator = container.persistentStoreCoordinator
    let storeDescription = container.persistentStoreDescriptions.first!

    // Exit current scope if persistentStore(for:) returns nil (migration is not needed).
    guard coordinator.persistentStore(for: oldStoreURL) != nil else {
        print("Migration not needed")
        return
    }

        // Replace one persistent store with another.
        do {
            try coordinator.replacePersistentStore(
                at: sharedStoreURL,
                withPersistentStoreFrom: oldStoreURL,
                type: .sqlite
            )
        } catch {
            fatalError("Something went wrong while migrating the store: \(error)")
        }

        // Delete old store.
        do {
            try coordinator.destroyPersistentStore(at: oldStoreURL, type: .sqlite, options: nil)
        } catch {
            fatalError("Something went wrong while deleting the old store: \(error)")
        }

        NSFileCoordinator(filePresenter: nil).coordinate(writingItemAt: oldStoreURL.deletingLastPathComponent(), options: .forDeleting, error: nil, byAccessor: { url in
            try? FileManager.default.removeItem(at: oldStoreURL)
            try? FileManager.default.removeItem(at: oldStoreURL.deletingLastPathComponent().appendingPathComponent("\(container.name).sqlite-shm"))
            try? FileManager.default.removeItem(at: oldStoreURL.deletingLastPathComponent().appendingPathComponent("\(container.name).sqlite-wal"))
            try? FileManager.default.removeItem(at: oldStoreURL.deletingLastPathComponent().appendingPathComponent("ckAssetFiles"))
        })

        // Unload the store and load it again with new storeDescription to re-enable CloudKit.
        if let persistentStore = container.persistentStoreCoordinator.persistentStores.first {
            do {
                try container.persistentStoreCoordinator.remove(persistentStore)
                print("Persistent store unloaded")
            } catch {
                print("Failed to unload persistent store: \(error)")
            }
        }

        // Set the URL of the storeDescription to the sharedStoreURL.
        storeDescription.url = sharedStoreURL
        // Modify the storeDescription to re-enable CloudKit integration.
        storeDescription.cloudKitContainerOptions = originalCloudKitOptions

        // Load the persistent store with the updated storeDescription.
        container.loadPersistentStores(completionHandler: { (storeDescription, error) in
            if let error = error as NSError? {
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        })

        print("Migration completed")
    }
}

That's the whole code. Now let's look at each part and I will explain to you what's happening and why.

Create a PersistenceController

You are probably familiar with the first part of the code. Especially if you (like me) start with the default Core Data stack in Xcode. When we start New Project and choose "App" and later check "Use Core Data" and "Host in iCloud", we will end up with something like this. PersistenceController struct, static let shared for our app's data and static var preview for our Preview Canvas.

In the for _ in block I am creating some objects for my previews. This is not important in the context of database migration, so please don't worry about this part of the code. In your app, this block will be different anyway.

struct PersistenceController {
    static let shared = PersistenceController()

    // Create a database for Preview Canvas.
    static var preview: PersistenceController = {
        let result = PersistenceController(inMemory: true)
        let viewContext = result.container.viewContext

        for _ in 0..<4 {
            let newGoal = CDEGoal(context: viewContext)
            newGoal.type = GoalType.number.rawValue
            newGoal.date = Date()
            newGoal.icon = "trophy"
            newGoal.name = "Goal name"
            newGoal.goal = 1000
            newGoal.currently = 500
            newGoal.status = GoalStatus.planned.rawValue
            newGoal.color = "purple"
            newGoal.id = UUID()
        }

        do {
            try viewContext.save()
        } catch {
            let nsError = error as NSError
            fatalError("Unresolved error \(nsError), \(nsError.userInfo)")
        }
        return result
    }()
// More code...
}

Declare properties

We need to declare a few properties first. We will use the name container for our NSPersistentCloudKitContainer. We will assign a particular iCloud container to it later in the init method.

We also declare oldStoreURL and sharedStoreURL properties that we will use later to move our database to the proper App Group folder.

let container: NSPersistentCloudKitContainer

var oldStoreURL: URL {
    let appSupport = FileManager.default.urls(
        for: .applicationSupportDirectory,
        in: .userDomainMask
    ).first!
    return appSupport.appendingPathComponent("YourAppName.sqlite")
}

// Define new App Group containerURL.
var sharedStoreURL: URL {
    let id = "group.com.yourDomain.YourAppName" // Use App Group's id here.
    let containerURL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: id)!
    return containerURL.appendingPathComponent("YourAppName.sqlite")
}

Create the container

We set our container as an NSPersistentCloudKitContainer with a name similar to "iCloud.com.yourDomain.YourAppName". One note here: while Apple suggests this naming convention, you can as well use a different one, for example, "iCloud.YourAppName". The most important thing is that this name must match the name of the iCloud container selected in the Signing & Capabilities tab of your app's target.

We also create a constant description that we will use later in the code. And a constant originalCloudKitOptions which we will pass to the migrateStore function later.

init(inMemory: Bool = false) {
    container = NSPersistentCloudKitContainer(name: "iCloud.com.yourDomain.YourAppName")

    let description = container.persistentStoreDescriptions.first!
    let originalCloudKitOptions = description.cloudKitContainerOptions

// Other code.
}

Check if migration is needed

In our code, we will check two times if the migration is needed. Here's the first time. We are checking if a database file exists at the oldStoreURL. This covers a situation when users run our app for the first time on some device. If so, the database doesn't exist yet on the device and we set the URL of our store to sharedStoreURL (App Group folder). Database migration won't be needed here at all.

init(inMemory: Bool = false) {
// Other code.

    // Use the App Group store if migration is not needed (if default store without App Group doesn't exist).
    if !FileManager.default.fileExists(atPath: oldStoreURL.path) {
        description.url = sharedStoreURL
    } else {
        // Disable CloudKit integration if migration is needed.
        description.cloudKitContainerOptions = nil
    }

// Other code.
}

As you can see, we also have an else statement. If a database file does exist on the device at the oldStoreURL, we disable iCloud sync by setting cloudKitContainerOptions to nil.

As I said earlier, my app was freezing during migration if iCloud was turned off on the device. With more detailed logging turned on, I saw that my app tries to connect to iCloud again and again and the migration code is not executed. The only solution I found is to turn off iCloud sync during the migration. So here in the code we are saying: if the database file exists at the oldStoreURL, turn off iCloud sync as we will perform the migration.

Load persistent stores and migrate data

We can migrate the database only if the persistent store was loaded, so we load persistent stores first and after this, we migrate the data.

init(inMemory: Bool = false) {
// Other code.

    // Load the persistent stores.
    container.loadPersistentStores(completionHandler: { (storeDescription, error) in
        if let error = error as NSError? {
            fatalError("Unresolved error \(error), \(error.userInfo)")
        }
    })

    // Perform the migration.
    migrateStore(for: container, originalCloudKitOptions: originalCloudKitOptions)

// Other code.
}

We perform the migration using the migrateStore function that we will write in a moment as part of our PersistenceController.

Function to migrate the store

In our PersistenceController we create a function named migrateStore which we execute at the end of our init method.

We will pass two arguments to this function: container and originalCloudKitOptions. We also declare the coordinator constant equal to container.persistentStoreCoordinator and storeDescription to make the part where we edit the container's options easier to read.

// Function migrateStore. Migrates data store to App Group if needed.
func migrateStore(for container: NSPersistentCloudKitContainer, originalCloudKitOptions: NSPersistentCloudKitContainerOptions?) {
    let coordinator = container.persistentStoreCoordinator
    let storeDescription = container.persistentStoreDescriptions.first!

    // More code.
}

Check if migration is needed (again)

We also write a guard statement to check for the second time if the migration is needed. Let's say that a user opened a previous version of our app on his device before. If persistentStore for oldStoreURL is different than nil, it means that the database at the default location exists and we need to migrate the data to the App Group folder. If it doesn't exist (which means that we already migrated our data), print "Migration not needed" and exit from the current scope (migrateStore function).

// Exit current scope if persistentStore(for:) returns nil (migration is not needed).
guard coordinator.persistentStore(for: oldStoreURL) != nil else {
    print("Migration not needed")
    return
}

Migrate database

With this code, we are migrating our database. But a more correct word would be replacing. And I will explain why in the next paragraphs.

// Replace one persistent store with another.
do {
    try coordinator.replacePersistentStore(
        at: sharedStoreURL,
        withPersistentStoreFrom: oldStoreURL,
        type: .sqlite
    )
} catch {
    fatalError("Something went wrong while migrating the store: \(error)")
}

We use the word "migrate" when we talk about changing one database to another (or when we change our data model). But this may lead us to one mistake. NSPersistentStoreCoordinator has 2 similar methods: migratePersistentStore and replacePersistentStore. There is one crucial difference between them:

As one anonymous Apple wrote on Apple Developer Forums:

migratePersistentStore is (...) creating a clean copy of all the data in your store file in a new location on the file system. Use replacePersistentStore instead to move the store to a new location.

I tested migratePersistentStore before knowing about replacePersistentStore and in the result I had duplicated data. With the replacePersistentStore method, no duplicates were created.

Delete old database

With the replacePersistentStore method, we replaced the store with our new store located in the App Group. Apple's documentation doesn't tell us much about it. It just says this: "Replaces one persistent store with another.". I thought this method will move one store to another place and will literally move the database files. But it doesn't. After running the code I noticed that a new version of the database was in fact created in the App Group folder, but old files were still present in the app support folder.

To get rid of old files, we need to delete them by ourselves. To do this, we will use destroyPersistentStore and removeItem methods.

// Delete old store.
do {
    try coordinator.destroyPersistentStore(at: oldStoreURL, type: .sqlite, options: nil)
} catch {
    fatalError("Something went wrong while deleting the old store: \(error)")
}

NSFileCoordinator(filePresenter: nil).coordinate(writingItemAt: oldStoreURL.deletingLastPathComponent(), options: .forDeleting, error: nil, byAccessor: { url in
    try? FileManager.default.removeItem(at: oldStoreURL) // Delete .sqlite file.
    try? FileManager.default.removeItem(at: oldStoreURL.deletingLastPathComponent().appendingPathComponent("\(container.name).sqlite-shm")) // Delete .sqlite-shm file.
    try? FileManager.default.removeItem(at: oldStoreURL.deletingLastPathComponent().appendingPathComponent("\(container.name).sqlite-wal")) // Delete .sqlite-wal file.
    try? FileManager.default.removeItem(at: oldStoreURL.deletingLastPathComponent().appendingPathComponent("ckAssetFiles")) // Delete ckAssetFiles.
    // ckAssetFiles files may be named like this: AppName_ckAssets
})

When I checked the Simulator's folder holding my database I saw 3 files there: .sqlite, .sqlite-shm and .sqlite-wal. I didn't see any ckAssetFiles files, but this may be due to iCloud being turned off in the Simulator. I decided to leave the last FileManager.default.removeItem as it was in the example. If there will be ckAssetFiles in the database folder, they will be deleted. If they are not there, nothing will happen.

I also need to mention that I tried to comment out coordinator.destroyPersistentStore and my files were not deleted. I tried to leave coordinator.destroyPersistentStore and comment out NSFileCoordinator and again - my old files were not deleted. We need both parts. We need to destroyPersistentStore first and later delete each file with FileManager.default.removeItem.

I found this solution on StackOverflow and I would like to quote a user named Jordan H who posted the solution:

In talking with an engineer in a WWDC lab, they explained it does not actually delete the database files at the provided location as the documentation seems to imply. It actually just truncates rather than deletes. If you want them gone you can manually delete the files (if you can ensure no other process or a different thread is accessing them).

Re-enable CloudKit

We moved our database. We deleted old files. Now we can re-enable CloudKit. As we can't change cloudKitContainerOptions on a running persistence store, we need to unload our current store, change storeDescription.cloudKitContainerOptions to the value previously stored in the originalCloudKitOptions constant and load the store again with these options.

// Unload the store and load it again with new storeDescription to re-enable CloudKit.
if let persistentStore = container.persistentStoreCoordinator.persistentStores.first {
    do {
        try container.persistentStoreCoordinator.remove(persistentStore)
        print("Persistent store unloaded")
    } catch {
        print("Failed to unload persistent store: \(error)")
    }
}

// Set the URL of the storeDescription to the sharedStoreURL
storeDescription.url = sharedStoreURL
// Modify the storeDescription to re-enable CloudKit integration
storeDescription.cloudKitContainerOptions = originalCloudKitOptions

// Load the persistent store with the updated storeDescription
container.loadPersistentStores(completionHandler: { (storeDescription, error) in
    if let error = error as NSError? {
        fatalError("Unresolved error \(error), \(error.userInfo)")
    }
})

print("Migration completed")

Final note

That's it. That's how I managed to migrate NSPersistentCloudKitContainer to App Groups. I tested this solution on Simulator and real devices. I tested with iCloud sync turned on and with iCloud sync turned off on the device. I tested if the data continues to sync after migration. And everything seems to work. However, I assume that it's not the best way of solving this problem. And also I am very far from being a Core Data expert, so please take this into consideration. Please double-check everything before using my code in your projects. Please test the results and make some changes if necessary. And if you know that something could be done better, please leave a comment and let me know about this.

If you want to support my work, please like, comment, share the article and most importantly...

📱Check out my apps on the App Store:
https://apps.apple.com/developer/next-planet/id1495155532

NEXT PLANET - UX and iOS dev