# Saved events to CSV for BI

Example version: 1.1, 27 September 2026. Python 3.9+, standard library only.
Status: runs locally with the included synthetic data. The API client is tested with mocked HTTPS responses in memory; no requests to an actual BOX installation have been made for this example.

## Result

Two JSON pages become a CSV containing four unique events. One exact duplicate is skipped. Import the table into a BI tool, Excel or LibreOffice and group it by camera and event type. A separate Python script produces the same grouped counts.

An optional `fetch_events.py` shows the next step with your data: your installation's Swagger → one selected API page → adapted `mapping.json` → CSV. It is not needed for the first run with training data.

The documented BOX 3.x contract places the list in `StoredViolationEvents.events`. Each event's fields come from the `StoredViolationEvent` schema of the particular installation. All `demo_*` fields and status values in the fixtures are invented for the exercise: they are **not the BOX API contract**. Their mapping is stored in `mapping.json`.

## Run

Extract the bundle and open a terminal in its directory:

```sh
python3 export_events.py sample-page-1.json sample-page-2.json --config mapping.json --output events.csv
python3 summarize.py events.csv --output summary.csv
python3 check_example.py
```

The first command prints `{"duplicates_skipped": 1, "rows": 4}`. `events.csv` must match `expected.csv`, and `summary.csv` must match `expected-summary.csv`. The event CSV has a UTF-8 BOM, comma separators and headers. In BI, treat ID, Camera, Type and Status as text. Time values carry an explicit UTC+03:00 offset; the script does not convert them. The conversion and summary commands replace their output files, so use new output names when retaining an earlier export.

## Inspect the result in BI

Import `events.csv`: rows = Camera, columns = Type, value = count of ID. Expected counts are Camera 1 / no_helmet = 2, Camera 2 / no_helmet = 1, and Camera 2 / vehicle = 1. These counts include confirmed, unreviewed and rejected records. Filter `Status = confirmed` to leave two records. This is a training illustration, not a site safety metric.

## Optional: retrieve one page through the API

First check `GET /api/statistics/event-storage/events/`, Bearer authentication, `page`, `page_size` and `sorting` in your build's Swagger. Your server determines numeric limits and the fields of individual events. Obtain a token using the normal method described in the API guide. This client does not sign in or refresh tokens.

The client accepts the origin and access token only through `BOX_ORIGIN` and `BOX_ACCESS_TOKEN`. In Bash or Zsh, set your installation's address and start hidden token input:

```sh
export BOX_ORIGIN='https://box.example.invalid'
read -r -s BOX_ACCESS_TOKEN
```

Replace the demonstration address with yours. After `read`, enter the access token without a Bearer prefix and press Enter; the input is not displayed. Then run:

```sh
export BOX_ACCESS_TOKEN
python3 fetch_events.py --page 1 --page-size 20 --sorting=-timestamp --timeout 15 --output raw-page-1.json
unset BOX_ACCESS_TOKEN
```

The request reads **only the selected page**. It has no date-range filters, does not traverse other pages or retry the request, and does not establish that an export is complete. Pass a leading minus as `--sorting=-timestamp`. Check other sort values in Swagger.

Success means HTTP 200, a valid JSON object and an `events` array; an empty array is accepted. The full response is saved as a new UTF-8 JSON file. The terminal prints only a summary, for example:

```json
{"events": 3, "page_requested": 1, "page_size_requested": 20, "status": "saved"}
```

`page_requested` and `page_size_requested` repeat the request parameters; they do not verify the meaning of the server's pagination fields. The file is never overwritten: choose another name. Diagnostics do not print the Authorization header, response body or network exception text.

HTTPS certificate-chain and hostname verification stay enabled; there is no option to disable them. For a private certificate authority, configure a trusted CA in the Python environment. Redirects are blocked, so use the final origin. The 15-second timeout applies to network operations, not the whole program. Responses are limited to 10 MiB; reduce the page size if needed. For HTTP 401/403, check the token and permissions; for 422, check the parameters in Swagger. An error returns exit code 1.

## Convert your own data

1. Retrieve one permitted page with `fetch_events.py`, or save JSON using your installation's normal client.
2. Inspect each event's schema. Copy `mapping.json` to `mapping-real.json` and replace the paths. Dotted paths such as `source.name` select nested objects. This simple mapper does not support arrays or keys containing literal dots.
3. Choose a stable `deduplicate_by` field. If the same key has different JSON, conversion stops; explicitly decide which event version to keep.
4. Pass the files in the desired order. The script does not check period/page completeness or sort events. It loads data into memory and is intended for small exports.

After adapting the fields:

```sh
python3 export_events.py raw-page-1.json --config mapping-real.json --output events-real.csv
```

The training mapping with `demo_*` fields cannot be used unchanged with real responses. Retain the column names `Camera` and `Type` if you want to use `summarize.py`.

Values that resemble a spreadsheet formula (`=`, `+`, `-` or `@` after whitespace) receive a leading apostrophe. This keeps them as safe text in spreadsheets, but changes their string representation; account for it during BI import. The script does not cast values to numeric types.

## Tests and scope

`check_example.py` checks exact CSV output, BI counts, duplicate handling, conflicting event versions, missing fields and formula escaping. Invalid input is detected before the CSV is written.

It also runs `check_fetch.py`. `unittest.mock` replaces HTTPS transport completely. Tests cover the GET path and parameters, Bearer header, TLS verification settings, a single request, empty events, JSON/HTTP errors, timeout, blocked redirects, no overwrite and absence of the token in diagnostics. Only a fake token, reserved `example.invalid` domains and synthetic responses are used; the tests make no network calls. A real TLS connection, actual user permissions, compatibility with a particular BOX build and BI import have **not been verified**.

Guides:
- https://viewlogic.ai/technology/box3-api/#event-response
- https://viewlogic.ai/technology/box3-api/#event-query
- https://viewlogic.ai/technology/event-contracts/

Included: `fetch_events.py`, `export_events.py`, `summarize.py`, `check_fetch.py`, `check_example.py`, `mapping.json`, two sample pages, `expected.csv` and `expected-summary.csv`. All fixtures are synthetic. Credentials are needed only for the optional request to your own installation.
