Why use SQList to generate Power BI reports from SharePoint lists, rather than using the Power BI native connector to SharePoint?

The main reason is that in practical terms using the native connector only works for small amount of data and simple queries. When you start using it on lists with many thousands of items and/or want to run queries across many large lists, you soon hit limitations in terms of amount of data you can query and the time it takes to execute them.

When you connect to a data source (e.g. a database, Excel file, or in this case SharePoint lists) from power BI you can choose two way of connecting to it:

  1. Direct query: in this type of connection, Power BI reads the data directly from the source every time you run a report. The advantages of this connection type are a) your report always shows up-to-date data, b) you offload some of the processing to the server providing the data instead of performing all the processing on the machine running Power BI; the only disadvantage is Power BI has to  connect to the data source every time is executes the report that can be affected by slow network connections (not really an issue in a business/corporate environment).
  2. Import: in this case, Power BI first imports all the data from the source into its own database, and then runs the report using that local copy of the data. The advantage of this connection is that, once all data is transferred from the source to the Power BI local database, you do not need to connect to the data source any longer; the disadvantages are a) the transfer of the data can take a long time when you want to report on large data sets, b) you must refresh all data imports every time you want an up-to-date report, c) all data processing is performed by the machine running Power BI which can be intensive in case of complex reports, d) there is a limit to the amount of data you can import (1Gb I think), e) there is a limit to the times you can refresh the data automatically.

AxioWorks SQList continuously export SharePoint lists and libraries as normalised SQL Server tables, making live SharePoint data available to reporting tools like Power BICrystal Reports, or SSRS.

The Power BI built-in connection to SharePoint only allows you to use the “Import” connection type.

Additional advantages of using SQList are:

  • SharePoint data is exported in a normalised form, which is the structure that allows you to execute the most flexible queries and, therefore, reports;
  • SQList can export SharePoint certain data in a format that would require complex configurations and time-consuming processing to obtain using the native connector (e.g. multi-value lookup columns, taxonomy metadata);
  • your SharePoint data is available as SQL Server tables, which gives you the option of overlaying it with SQL views that can greatly enhance access control and performance when generating the reports;
  • you can use “Direct query” connections (in fact, we recommend them) for up-to-date reports that executes very fast as the heavy processing is performed by SQList (which is built for that).
  • configuring SQList can take as little as 5 minutes to have your data available in SQL Server, there is virtually no learning curve and not changes to your environment are necessary;
  • virtually all reporting and BI tools can connect directly to SQL Server tables so all the above is valid for them too (SSR, Crystal Reports, Cognos, Tableau, and so on).

Finally, there are some features that are available in Power BI when using “Import” rather than “Direct query” (this is good summary https://powerbidocs.com/2021/02/21/power-bi-import-vs-direct-query-mode-difference/), but that can be easily replaced by equivalent (and even more powerful) functions in the SQL Server side.