Power BI Monitoring: Activity Logs and Asset Management with Python and Fabric

—

by

Alejandro Serrano
in ,

For organizations, getting a clear view of platform resources and how they’re being used is key. This usually boils down to two main things:

  1. Visibility of Assets and Access: A simple way to see all the assets created on the platform and who (users or groups) has access to them.
  2. Usage Monitoring: Keeping an eye on asset usage to ensure compliance and optimize operations.

If you’ve worked with Power BI, you might know it only retains usage logs for 90 days by default. That’s a pretty short window for analyzing trends and behaviors. Plus, getting and reporting on this data isn’t exactly straightforward since it’s mostly accessible through the Power BI REST APIs.

A while ago, I came across a fantastic Power BI REST API Connector created by Miguel Escobar:
https://www.thepoweruser.com/2021/02/21/power-bi-rest-api-connector/

Even though the code hasn’t been updated in a while, this connector is still a great starting point for exploring your environment and pulling audit data directly into Power BI. (If you’re curious about which APIs are being called, you can always take a look at the PBIAPI.pq file.)

I’ve used this connector in several projects, and it helped me get up and running pretty quickly. However, it doesn’t handle historization, which was still an issue for me.

Recently, Microsoft released a cool new feature for Fabric: the ability to run Python code in Notebooks without using Spark. Around the same time, I found some code from PBI Guy on how to query Power BI Activity Logs with Python. It felt like the perfect time to experiment!

Power BI and Activity Logs with Python – PBI Guy

Here’s what I did step by step:

Set up access:

    • Created a group in Microsoft Entra ID to grant access to the Power BI Admin APIs.
    • Granted this group read-only access to the admin APIs in the Power BI admin portal.
    • Created an app in Microsoft Entra ID to use for API access, added it to the group, and generated the required client secret.

Adapted the code:

    • Took PBI Guy’s code as a reference, pasted it into a Fabric Notebook, and switched the language to Python.
    • Made changes to let the ActivityEvents API accept a range of dates (since it only works for one day at a time).
    • Updated the code to write the output as Parquet files directly to the lakehouse instead of CSV.
    • Added some exception handling to make it more robust.

Processed the data:

    • Ran the updated code and created Parquet files containing the audit log data.
    • Used a PySpark Notebook to create a managed table from the Audit Log folder for easy access.

Connected to Power BI:

    • Created a Lakehouse connection in Power BI to access the new Audit Log table for reporting.

This experiment tackled my immediate need to retain audit history. But really, it’s just the starting point. The next step is to use other Admin APIs to pull detailed info on all Power BI assets and their access controls.

The good news? You can start simple by using Miguel Escobar’s REST API Connector to quickly explore what’s relevant. Once you have a clear picture, you can dive deeper into custom coding to meet your specific requirements.