PyIceberg

Below is an example of using PyIceberg ↗ to connect to R2 Data Catalog.

Prerequisites

• Sign up for a Cloudflare account ↗.
• Create an R2 bucket and enable the data catalog.
• Create an R2 API token with both R2 and data catalog permissions.
• Install the PyIceberg ↗ and PyArrow ↗ libraries.

Example usage

Python

import pyarrow as pa
from pyiceberg.catalog.rest import RestCatalog
from pyiceberg.exceptions import NamespaceAlreadyExistsError

# Define catalog connection details (replace variables)
WAREHOUSE = "<WAREHOUSE>"
TOKEN = "<TOKEN>"
CATALOG_URI = "<CATALOG_URI>"

# Connect to R2 Data Catalog
catalog = RestCatalog(
    name="my_catalog",
    warehouse=WAREHOUSE,
    uri=CATALOG_URI,
    token=TOKEN,
)

# Create default namespace
catalog.create_namespace("default")

# Create simple PyArrow table
df = pa.table({
    "id": [1, 2, 3],
    "name": ["Alice", "Bob", "Charlie"],
})

# Create an Iceberg table
test_table = ("default", "my_table")
table = catalog.create_table(
    test_table,
    schema=df.schema,
)

Explain Code