type any.
Table.AddColumn to set it to number
[Num] * 2 work?Powerquery does not know what type will be returned by your function. That’s because each is by definition a function that returns type any
each expression in the Lexical Grammareach functions
Facebook.Graph (on my machine)#shared to a table instead of calling Record.FieldNames() )#sharedTo get a list of all identifiers (functions, variables, constants) I use the variable named #shared . Create a new blank query, then paste this
let
IdentifierList = List.Sort(Record.FieldNames( #shared ))
in
IdentifierListI copy using Copy Entire List, then storing the results into a PowerShell variable. I Repeat the same with Power BI.
To find out exactly which functions are different, use the Power Shell operator -NotIn
# Find functions in PBI but not Excel
$MissingExcel = $PowerBI | ? { $_ -notin $Excel }
# Find functions in Excel but not PBI
$MissingPowerBI = $Excel | ? { $_ -notin $PowerBI }Facebook.GraphNote: This is the list for today, on my machine. Run the #shared query to find any changes.
AI.ExecuteInProc
AI.ExecuteInternal
AI.ExternalSort
AI.GetAutoMLEntity
AI.SampleStratifiedWithHoldout
AI.TestConnection
AIFunctions.Capacities
AIFunctions.Contents
AIFunctions.ExecuteInternal
AIFunctions.GetAutoMLEntity
AIFunctions.PostProcess
AIInsights.Contents
AIInsights.ContentsGenerator
AML.Execute
AML.ExecuteBatch
Acterys.Contents
Actian.Contents
AmazonRedshift.Database
Anaplan.Contents
ApacheHiveLLAP.Database
ApacheSpark.Tables
Asana.Tables
AtScale.Cubes
AutomationAnywhere.Feed
AzureCostManagement.Contents
AzureCostManagement.Tables
AzureDataExplorer.Contents
AzureDataExplorer.Databases
AzureDevOpsServer.AccountContents
AzureDevOpsServer.AnalyticsViews
AzureDevOpsServer.Feed
AzureDevOpsServer.Views
AzureEnterprise.Contents
AzureEnterprise.Tables
AzureHiveLLAP.Database
AzureMLFunctions.Contents
AzureMLFunctions.Execute
AzureMLFunctions.ExecuteBatch
AzureSpark.Tables
AzureTimeSeriesInsights.Contents
BI360.Contents
BIConnector.Contents
Binary.Range
Cdm.Contents
Cdm.MapToEntity
Cds.Contents
Cds.Entities
Cherwell.SavedSearches
Cognite.Contents
CommonDataService.Database
Compression.Brotli
Compression.LZ4
Compression.None
Compression.Snappy
Compression.Zstandard
CustomerInsights.Contents
DataVirtuality.Database
DataWorld.Contents
DataWorld.Dataset
Databricks.Contents
Denodo.Contents
DocumentDB.Contents
Dremio.Databases
Dynamics365BusinessCentral.Contents
Dynamics365BusinessCentral.EnvironmentContents
Dynamics365BusinessCentralOnPremises.Contents
DynamicsNav.Contents
Emigo.Contents
Emigo.GetExtractFunction
EmigoDataSourceConnector.GetExtractFunction
EmigoDataSourceConnector.NavigationFunctionType
EntersoftBusinessSuite.Contents
Essbase.Cubes
Exasol.Database
FactSetAnalytics.AuthenticationCheck
FactSetAnalytics.Functions
Fhir.Contents
Foundry.Contents
Geography.FromWellKnownText
Geography.ToWellKnownText
GeographyPoint.From
Geometry.FromWellKnownText
Geometry.ToWellKnownText
GeometryPoint.From
Github.Contents
Github.PagedTable
Github.Tables
GoogleAnalytics.Accounts
GoogleBigQuery.Database
Graph.Nodes
HexagonSmartApi.ApplySelectList
HexagonSmartApi.ApplyUnitsOfMeasure
HexagonSmartApi.ExecuteParametricFilterOnFilterRecord
HexagonSmartApi.ExecuteParametricFilterOnFilterUrl
HexagonSmartApi.Feed
HexagonSmartApi.GenerateParametricFilterByFilterSourceType
HexagonSmartApi.GetODataMetadata
HexagonSmartApi.Typecast
HiveProtocol.HTTP
HiveProtocol.Standard
HiveProtocol.Type
Html.Table
IRIS.Database
Impala.Database
Indexima.Database
IndustrialAppStore.NavigationTable
InformationGrid.Contents
Intune.Contents
JamfPro.Contents
JethroODBC.Database
Kyligence.Database
Linkar.Contents
LinkedIn.SalesContracts
LinkedIn.SalesContractsWithReportAccess
LinkedIn.SalesNavigator
LinkedIn.SalesNavigatorAnalytics
LinkedIn.SalesNavigatorAnalyticsImpl
List.ConformToPageReader
MailChimp.Collection
MailChimp.Instance
MailChimp.Tables
MailChimp.TablesV2
MariaDB.Contents
MarkLogicODBC.Contents
Marketo.Activities
Marketo.Leads
Marketo.Tables
MicroStrategyDataset.Contents
MicroStrategyDataset.TestConnection
MicrosoftAzureConsumptionInsights.Contents
MicrosoftAzureConsumptionInsights.Tables
MicrosoftAzureConsumptionInsights.Test
MicrosoftGraphSecurity.Contents
Mixpanel.Contents
Mixpanel.Export
Mixpanel.FunnelById
Mixpanel.FunnelByName
Mixpanel.Funnels
Mixpanel.Segmentation
Mixpanel.Tables
Netezza.Database
PQ_Hi_World.Contents
Parquet.Document
Paxata.Contents
Pdf.Tables
PlanviewEnterprise.CallQueryService
PlanviewEnterprise.Feed
PlanviewProjectplace.Contents
PowerBI.Dataflows
PowerPlatform.Dataflows
ProductInsights.Contents
ProductInsights.QueryMetric
Projectplace.Feed
Python.Execute
QubolePresto.Contents
QuickBase.Contents
QuickBooks.Query
QuickBooks.Report
QuickBooks.Tables
QuickBooksOnline.Tables
R.Execute
Roamler.Contents
ShortcutsBI.Contents
Siteimprove.Contents
Smartsheet.Content
Smartsheet.Query
Smartsheet.Tables
Snowflake.Databases
Spark.Tables
SparkPost.GetList
SparkPost.GetTable
SparkPost.NavTable
SparkProtocol.Azure
SparkProtocol.HTTP
SparkProtocol.Standard
SparkProtocol.Type
Spigit.Contents
StarburstPresto.Contents
Stripe.Contents
Stripe.Method
Stripe.Tables
SurveyMonkey.Contents
SweetIQ.Contents
SweetIQ.Tables
Table.AddFuzzyClusterColumn
Table.CombineColumnsToRecord
Table.ConformToPageReader
Table.FuzzyGroup
TeamDesk.Database
TeamDesk.Select
TeamDesk.SelectView
Tenforce.Contents
TibcoTdv.DataSource
TimeSeriesInsights.Contents
Troux.CustomFeed
Troux.Feed
Troux.TestConnection
Twilio.Contents
Twilio.Tables
Twilio.URL
VSTS.AccountContents
VSTS.AnalyticsViews
VSTS.Contents
VSTS.Feed
VSTS.Views
Value.Alternates
Value.Expression
Value.Lineage
Value.Optimize
Value.Traits
Vena.Contents
Vertica.Database
VesselInsight.Contents
Web.BrowserContents
Webtrends.KeyMetrics
Webtrends.Profile
Webtrends.ReportContents
Webtrends.Tables
WebtrendsAnalytics.Tables
Witivio.Contents
WorkforceDimensions.Contents
WorkplaceAnalytics.Data
Zendesk.Collection
Zendesk.Tables
date, time, datetime, datetimezone and durationprimative typesFacetsWeb.Content‘s Query and RelativePath parametersWeb.Contents() refresh errors all the way back in 2016!
advanced editor check this out.VS Code extension: Power Query uses PowerQuery-Parser and PowerQuery-Formatter
Query Diagnostics in Power BI
copy -> paste multiple queries into a new Power BI Report (notice it included a required referenced query ListAsText even though I didn’t select it.
copy -> paste multiple queries into at text editor
What do you do when text files are not formatted as .csv or .json ? If they repeat a pattern, you can automate it.
( You can download the final .PBIX report here )
This file always uses:
: in it. If you skip all lines without a : it filters all unwanted linesTitle
---------------------
DeviceID: One
Info1: One:Info1
Info2: One:Info2
---------------------
DeviceID: Two
Info1: Two: Info1
Info2: Two: Info2
---------------------
DeviceID: DeviceID
Info1: Three: Info1
Info2: Three: Info2First take a look at the final function. Then I’ll explain how it works.
The main query calls a custom function
ConvertTableFromTextLines( "c:\data\sample.txt", ":", 3 )All the work is one here
let ConvertTableFromText = (filepath as text, splitCharacter as text, linesPerRecord as number, optional encoding as nullable number) as table =>
let
TextLines = Lines.FromBinary(
File.Contents( filepath, null ),
null,
null,
encoding ?? TextEncoding.Utf8
),
TextLineTable = Table.FromColumns(
{ TextLines }, {"Line"}
),
Pairs = Table.SelectRows(
TextLineTable,
each Text.Contains( [Line], splitCharacter )
),
SingleRecordAsCols = Table.SplitColumn(
Pairs,
"Line",
Splitter.SplitTextByEachDelimiter(
{ splitCharacter }, QuoteStyle.Csv, false
),
{"Name", "Value"}
),
TotalLines = Table.RowCount( SingleRecordAsCols ) ,
NumberOfGroups = TotalLines / linesPerRecord,
ListOfRecords = List.Generate(
() => 0,
each _ < NumberOfGroups,
each _ + 1,
each (
Record.FromTable(
Table.Range(
SingleRecordAsCols,
_ * linesPerRecord,
linesPerRecord
)
)
)
),
t = Table.FromList(
ListOfRecords, Splitter.SplitByNothing(), null, null, ExtraValues.Error
),
columnNameList = Record.FieldNames( ListOfRecords{0} ),
TableOfRecords = Table.ExpandRecordColumn(
t, "Column1", columnNameList, columnNameList
)
in
TableOfRecords
in
ConvertTableFromText
First we load the text file as a list. File.Contents( filepath ) reads all file types. Lines.FromBinary( Contents ) converts it into text. Depending on how the file was saved, you may need to set the encoding
Don’t worry if encoding sounds scary. If text loads with the wrong characters, try the other one. Two good ones to try first are:
TextEncoding.Windows | Power BI defaults to this for windows files |
TextEncoding.Utf8 | Anything on the internet uses UTF8 by default. |
ASCII then UTF8 will workNext remove any lines that are missing :
Then split each line into two columns:
Transform Column -> Text -> Split -> By Delimiter
I started with these options
I renamed them to Name and Value because they will be converted into records.
To convert multiple lines into a single recordI used List.Generate. You can think of it like a for loop
There are
We need to loop 3 times. I’ll recreate a loop like this, converted to Power Query
for( $i = 0 ; $i < NumberOfGroups; $i += 1){
CreateRecord()
}List.Generate() has 4 arguments. Each one is a function. It uses:
initialize sets your starting condition. ( set i to 0 )condition tests if we’ve reached the total number of groupsnext changes the value of i. We want to add 1selector function creates the value every loop. This calls CreateRecord() once for every loop ListOfRecords = List.Generate(
() => 0,
each _ < NumberOfGroups,
each _ + 1,
each (
Record.FromTable(
Table.Range(
SingleRecordAsCols,
_ * linesPerRecord,
linesPerRecord
)
)
)
),The next function first grabs rows from the table SingleRecordAsCols. It converts those into a single record
Every loop 3 is added to the offset.
i value | offset | count |
| 0 | 0 * 3 = 0 | 3 |
| 1 | 1 * 3 = 3 | 3 |
| 2 | 2 * 3 = 6 | 3 |
You end up with a list of records
Finally expand records to columns
The final result:
