bidshelper Wiki & Documentation Rss Feed

New Comment on "Design Warnings"

talinkio — Fri, 19 Apr 2013 20:23:11 GMT

First, just got this and it is great. Thank you. I was testing out the design warning feature and it is throwing false positives for the 2012 version. It has a warning for all of my DB connections which indeed are coming from a config file. They even have the little blue corner mentioning they are from a configuration. Thank you.

New Comment on "Deploy SSIS Packages"

SNTKAVIN — Fri, 12 Apr 2013 16:30:40 GMT

I see lots of useful functions which I was looking from SSIS. Thanks a lot. But I do not understand the difference between "SQL Server Destination" and "SSIS Package Store (MSDB) ". I tried to find out but I couldn't . Please explain. Thanks

Updated Wiki: xcopy deploy

furmangg — Fri, 12 Apr 2013 10:20:52 GMT

To do an "xcopy deploy" you simply unzip the "xcopy deploy" zip file to either

<My Documents>/Visual Studio 2005/Addins (for SQL Server 2005)
or
<My Documents>/Visual Studio 2008/Addins (for SQL Server 2008 and SQL Server 2008 R2)
or
<My Documents>/Visual Studio 2010/Addins (for SQL Server 2012 in Visual Studio 2010)
or
<My Documents>/Visual Studio 2012/Addins (for SQL Server 2012 in Visual Studio 2012)

If the Addins directory doesn't exist already, create it.

This lets your run BIDS Helper in situations where you do not have administrator rights on the machine in order to be able to run the full installer.

For SQL Server 2012, you must also unblock every file in the Addins directory:

(If you don't unblock the DLLs, you will get the error message: "The Add-in 'BIDSHelper' failed to load or caused an exception. Error number: 80131515")

After restarting Visual Studio, check Help... About Microsoft Visual Studio and make sure you see BIDS Helper. If you don't, you may need to go to Tools... Options... Add-in/Macro Security... then add the above path to the list, then restart Visual Studio. If you see BIDS Helper but it is not functioning, look at the Tools... Options... BIDS Helper... Version tab for more information about the problem.

To fully configure the Biml Package Generator plugin as part of an xcopy deployment, please see Manually Configuring Biml Package Generator.

Updated Wiki: Version Notification

furmangg — Thu, 11 Apr 2013 19:30:55 GMT

Version Notification

BIDS Helper is constantly adding new features and fixing recently discovered bugs. We encourage everyone to use the latest version. The Version Notification feature helps you stay current by alerting you when a new BIDS Helper version is released. A balloon and icon appear in the system tray. Click the balloon and BIDS Helper will open Internet Explorer and take you to the download page for the latest release.

If you do not have time at that moment to install the latest BIDS Helper, right click on the BIDS Helper icon and choose to have BIDS Helper remind you later (in 7 days).

If you are not interested in installing that particular version, but you wish to be notified about future releases, you can dismiss the notification about that particular version.

BIDS Helper checks to see that you have the latest version once a week. If you ever want to validate that you have the latest version installed, go to the Tools menu and choose Options then expand the BIDS Helper tree, then click Version. When that Version screen opens, it shows you the version of BIDS Helper you have installed and also checks to make sure you have the latest installed:

Starting with BIDS Helper 1.6.3, the Version screen also displays some information about your version of SQL Server development tools, your version of Visual Studio, and any problems that occurred trying to start BIDS Helper:

For example...

"SSDTBI 2012 for Visual Studio 2012 was detected" means that you have installed the SQL Server 2012 developer tools (properly named SSDTBI) and that you are currently in Visual Studio 2012.
"BIDS 2008 R2 for Visual Studio 2008 was detected" means that you have installed the SQL Server 2008 R2 developer tools (named BIDS) and that you are currently in Visual Studio 2008.
"SSDTBI for Visual Studio 2012 was NOT detected. BIDS Helper disabled." means that you do not have the SQL Server developer tools installed for Visual Studio 2012 so BIDS Helper is not running.
"BIDS for Visual Studio 2008 was NOT detected. BIDS Helper disabled." means that you do not have the SQL Server developer tools installed for Visual Studio 2008 so BIDS Helper is not running.

Updated Wiki: Version Notification

furmangg — Thu, 11 Apr 2013 19:30:26 GMT

Version Notification

"SSDTBI 2012 for Visual Studio 2012 was detected" means that you have installed the SQL Server 2012 developer tools (properly named SSDTBI) and that you are currently in Visual Studio 2012.
"BIDS 2008 R2 for Visual Studio 2008 was detected" means that you have installed the SQL Server 2008 R2 developer tools (named BIDS) and that you are currently in Visual Studio 2008.
"SSDTBI for Visual Studio 2012 was NOT detected. BIDS Helper disabled." means that you do not have the SQL Server developer tools installed for Visual Studio 2012 so BIDS Helper is not running.
"BIDS for Visual Studio 2008 was NOT detected. BIDS Helper disabled." means that you do not have the SQL Server developer tools installed for Visual Studio 2008 so BIDS Helper is not running.

Updated Wiki: Samples and Tutorials

furmangg — Thu, 11 Apr 2013 15:21:37 GMT

Tutorials on Biml and BimlScript

Videos

Introduction to Biml in BIDS Helper

Other Resources

More tutorials and examples are available from bimlscript.com

Updated Wiki: Samples and Tutorials

furmangg — Thu, 11 Apr 2013 15:19:57 GMT

Tutorials on Biml and BimlScript

Videos

Introduction to Biml in BIDS Helper

Updated Wiki: Creating Project Connection Managers Using Biml

furmangg — Thu, 11 Apr 2013 15:17:54 GMT

This article is going to walk through the process of creating an SSIS 2012 package which uses a project connection manager using Biml and the Biml Package Generator feature in BIDS Helper.

This feature is available in BIDS Helper 1.6.3 and later and in SSIS 2012 projects in project deployment mode (not in legacy deployment mode). The code for this feature was generously created for the community by Davide Mauri.

For this example, copy and paste the following Biml into the a .biml document, as seen in the other examples.

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
  <Connections>
    <OleDbConnection Name="OLTP" ConnectionString="Data Source=localhost;Initial Catalog=tempdb;Provider=SQLNCLI11.1;Integrated Security=SSPI;Auto Translate=False;" CreateInProject="false"/>
    <OleDbConnection Name="OLTP2" ConnectionString="Data Source=localhost;Initial Catalog=tempdb;Provider=SQLNCLI11.1;Integrated Security=SSPI;Auto Translate=False;" CreateInProject="true"/>
  </Connections>
  <Packages>
    <Package Name="Test2" ConstraintMode="Linear">
      <Connections>
        <Connection ConnectionName="OLTP2"/>
      </Connections>
      <Tasks>
        <ExecuteSQL Name="ES Test A" ConnectionName="OLTP2">
          <DirectInput>SELECT Test=1;</DirectInput>
        </ExecuteSQL>
        <ExecuteSQL Name="ES Test B" ConnectionName="OLTP">
          <DirectInput>SELECT Test=1;</DirectInput>
        </ExecuteSQL>
        <Dataflow Name="DF Test">
          <Transformations>
            <OleDbSource ConnectionName="OLTP2" Name="Source">
              <DirectInput>SELECT Test=1;</DirectInput>
            </OleDbSource>
            <OleDbDestination ConnectionName="OLTP" Name="Dest">
              <ExternalTableOutput Table="dbo.Dest"></ExternalTableOutput>
            </OleDbDestination>
          </Transformations>
        </Dataflow>
      </Tasks>
    </Package>
  </Packages>
</Biml>

Note the CreateInProject="true" setting in red on the connection. This signals Biml to create a project level connection manager.

New Comment on "Create Fixed Width Columns"

AHPerez — Thu, 11 Apr 2013 14:13:26 GMT

I've been doing this by creating a FlatFile connection with 2 columns, then using the concatenate formula in XL with my column names and widths to create the rest of the columns (usually at least 200 for new vendor imports), then manually pasting this into the dtsx file . That was a huge time saver over the frustration of doing it manually. Your solution is far easier than that. Thanks very much for such a useful and elegant bit of programming.

New Comment on "Create Fixed Width Columns"

AHPerez — Thu, 11 Apr 2013 14:11:11 GMT

I've been doing this by creating a FlatFile connection with 2 columns, then using the concatenate formula in XL with my column names and widths to create the rest of the columns (usually at least 200) for new vendor imports. That was a huge time saver over the frustration of doing it manually. Your solution is far easier than that. Thanks very much for such a useful bit of programming.

Updated Wiki: Test Aggregation Performance

furmangg — Sun, 07 Apr 2013 01:46:28 GMT

Test Aggregation Performance

If the ProcessIndexes phase of cube processing is exceeding the processing window, the aggregation design may simply be too heavy. One approach is to identify large aggregations which only slightly improve query performance compared to an agg miss, then delete those aggregations. Another approach is to identify aggregations that can pull double duty enabling you to delete other aggregations with little query performance loss.

From Aggregation Manager, you can launch this feature by right-clicking on the cube node or on an aggregation design node:

The performance tests can be run with several options.

The first option is labeled "query performance with that aggregation". BIDS Helper generates an MDX query for each aggregation. The query utilizes the Exists-with-a-measure-group function instead of querying a measure in that measure group. This technique is used to avoid interference from the MDX script and to avoid semi-additive measures forcing the storage engine to use the date dimension granularity (instead of the aggregation granularity). When the query is run, the Execute MDX Script event duration is timed, then the rest of the query is timed. This option is fairly fast since queries that hit aggs should be fast.

The second option is labeled "query performance with no aggregations". BIDS Helper begins a transaction (so that these changes will only be visible to BIDS Helper and no other user sessions, and so these changes can be rolled back), then it deploys an empty aggregation design over the old one and runs ProcessIndexes. Then each MDX query generated above is run and timed to quantify query performance in the event of an agg miss. This option is fairly slow since each query will miss all aggs.

The third option is labeled "query performance with some aggregations". BIDS Helper utilizes the Similar Aggregations code to identify aggregations that can be deleted causing other aggregations to be used in their place. For instance, if you have one agg at the Quarter level and one at the Month level, you could delete the Quarter agg and Analysis Services will use the Month agg in its place with slight performance loss. To test the performance loss, BIDS Helper begins a transaction, then progressively deletes more and more aggs, running the MDX queries for the aggs it deletes to determine which agg is used instead. This option is the slowest since it has to run ProcessIndexes between each test.

Once the tests are completed, a report opens:

Page 1

Page 1 of the report has the following columns:

Hit Agg? shows whether the constructed MDX query hit this agg during test #1.
Rows, if it appears on an aggregation row, shows the rowcount of the aggregation across all partitions in this aggregation design. On the summary row for the aggregation design, this column shows the rowcount of the fact data across all partitions in this aggregation design. On the summary row for the measure group, this column shows the rowcount of all partitions in the measure group which have processed aggregations.
Approx Size shows the size of the aggregation as a percentage of the size of the partitions in this aggregation design. Aggregations can be smaller (and thus, more efficient to read from) because they have fewer rows and fewer dimension keys. While the previous column (Rows) shows just the rowcount, the Approx Size column gives a more accurate feel for the size of an aggregation by accounting for the number of dimension keys in that aggregation.
MDX Script With Agg tests the performance of the cube when all aggregations are processed. It shows the duration of the Execute MDX Script event in seconds. This column helps determine how quickly named sets and scope statements in the MDX script get resolved when the first user hits the cube.
MDX Script Without Aggs tests the performance of the cube when no aggregations are processed. It shows the duration of the Execute MDX Script event in seconds. This column helps determine how slowly (compared to MDX Script With Agg) named sets and scope statements in the MDX script get resolved when the first user hits the cube. This column is blank if you do not check the "query performance with no aggregations" option.
Query With Agg tests the performance of the generated MDX query when that particular aggregation is processed. It shows the duration of the MDX query in seconds.
Query Without Aggs tests the performance of the generated MDX query when no aggregations are processed. It shows the duration of the MDX query in seconds. This column is blank if you do not check the "query performance with no aggregations" option.

In page 1, notice that the "Query 0 1 Store Inventory" aggregation helps a query run in 12.55 seconds, but that same query runs in 13.58 seconds without any aggregations. This aggregation probably does not provide significant enough lift to justify the extra storage and processing time it requires. So it can be deleted with reasonable confidence that query performance will not suffer significantly.

If you check the "query performance with some aggregations" option, page 2 and 3 of the report contain information.

Page 2

Page 2 of the report has the following columns:

Aggregation shows the name of the aggregation being tested. The MDX query for this aggregation is run against an aggregation design without that aggregation, and the aggregation used in its place is determined.
Similar Aggregation Used shows the aggregation used in place of the aggregation which is deleted.
MDX Script Diff shows the difference (versus the MDX Script With Agg column on the first page) when this particular aggregation is deleted and Similar Aggregation Used is used in its place.
MDX Script Diff % shows the difference as a percentage (versus the MDX Script With Agg column on the first page) when this particular aggregation is deleted and Similar Aggregation Used is used in its place.
Query Diff shows the difference (versus the Query With Agg column on the first page) when this particular aggregation is deleted and Similar Aggregation Used is used in its place.
Query Diff % shows the difference as a percentage (versus the Query With Agg column on the first page) when this particular aggregation is deleted and Similar Aggregation Used is used in its place.

In page 2, notice the report says "Delete: Interactive ALL-ALL-ALL Store Inventory." This can be read as follows. If you delete the "Interactive ALL-ALL-ALL Store Inventory" aggregation, then the "Query 8 Store Inventory" aggregation will be used in its place and the difference in query performance will be 0% (too small to measure). Continue reading. Notice the report says "Delete: Interactive ALL-ALL-ALL Store Inventory, Query 8 Store Inventory." This can be read as follows. If you delete the "Interactive ALL-ALL-ALL Store Inventory" aggregation and the "Query 8 Store Inventory" aggregation, the "Interactive ALL-ALL-ALL Store Inventory" generated query will use the "Query 14 15 Store Inventory" aggregation instead and the "Query 8 Store Inventory" generated query will use the "Query 13 Store Inventory" aggregation. Down a little further, note that deleting the "Query 14 15 Store Inventory" aggregation will use "Query 0 1 Store Inventory" in its place, but the query will run 90.89% slower (though that 90% is less than a second). Aggregations not listed on this page will cause a complete agg miss, and the query performance difference for an agg miss is shown on the first page in the Query Without Aggs column. Note that "Query 13 Store Inventory" is the name of the aggregation. The actual Project REAL MDX query called "#13" is not known or used by BIDS Helper. BIDS Helper simply generates an MDX query designed to hit that aggregation.

Page 3

Page 3 of the report is the exact same except that it groups by the aggregation being tested so that you can see, as a series of aggs are deleted, which aggregation is used in place of the aggregation being tested. It also shows a Process Agg Time column which displays the number of seconds it took to ProcessIndexes the partitions in this aggregation design with this aggregation versus how long it too to ProcessIndexes without this aggregation.

Download the sample report seen in the above screenshots. It was generated against the Project REAL cube.

Note:

Even though no permanent changes to the cube are made (because everything is done in a transaction and then rolled back), it is still not recommended you run this feature on your production server since BIDS Helper will clear the cache before every MDX query it tests.
Starting with BIDS Helper 1.6.3, Test Aggregation Performance will clear the file system cache if ASSP version 1.3.5 or later is installed on the SSAS server.
If you choose to delete aggregations, be sure to retest your actual query workload under single user and multi-user scenarios to ensure the changes have not significantly impacted query response time or scalability. The MDX query used to test each aggregation is a generated MDX query that may not be representative of real user query patterns.

Updated Wiki: Tabular Annotation Workaround

furmangg — Wed, 20 Mar 2013 17:46:08 GMT

Tabular Annotation Workaround

After upgrading to SQL Server 2012 SP1 SSDT, some users have complained about receiving an error that prevents them from opening the .bim file in a Tabular model:

============================
Error Message:
============================

An error occurred while opening the model on the workspace database. Reason: ReadElementContentAs() methods cannot be called on an element that has child elements.

============================
Call Stack:
============================

 at Microsoft.AnalysisServices.VSHost.VSHostManager.PrepareSandbox(Boolean newProject, Boolean& isRefreshNeeded, Boolean& isImpersonationChanged, Boolean& saveRequired, List`1& truncatedTables, Boolean isRealTimeMode, Int32 clientCompatibilityLevel)
 at Microsoft.AnalysisServices.VSHost.Integration.EditorFactory.CreateEditorInstance(UInt32 grfCreateDoc, String pszMkDocument, String pszPhysicalView, IVsHierarchy pvHier, UInt32 itemid, IntPtr punkDocDataExisting, IntPtr& ppunkDocView, IntPtr& ppunkDocData, String& pbstrEditorCaption, Guid& pguidCmdUI, Int32& pgrfCDW)

============================

This problem is a bug in SSDT in that it does not properly parse annotations created by BIDS Helper.

UPDATE: This SSDT bug has been fixed in SQL2012 SP1 CU3.

If you are experiencing this issue you have two options:

1. Install SQL2012 SP1 CU3
2. Install BIDS Helper 1.6.2 and use a workaround utility which changes the storage format for BIDS Helper annotations. BIDS Helper release 1.6.2 includes this workaround feature described below.

If you choose to use the BIDS Helper workaround utility instead of installing CU3, do the following. With the .bim file closed, right click on the .bim file and select Tabular Annotation Workaround...

This feature will popup a confirmation dialog. Read it carefully, especially the note about every developer on your team upgrading to the latest version of BIDS Helper.

As background on this issue, see the Connect issue here.

New Comment on "Expression List"

ktrock — Thu, 14 Mar 2013 19:20:21 GMT

That's neat, though I notice you must have a project open in order to have View -->Other Windows--> Expression List available. If you're just looking at a .dtsx you don't get that option in 1.6.2 for SQL 2005.

Updated Wiki: Tabular Annotation Workaround

furmangg — Wed, 27 Feb 2013 03:58:58 GMT

Tabular Annotation Workaround

After upgrading to SQL Server 2012 SP1 SSDT, some users have complained about receiving an error that prevents them from opening the .bim file in a Tabular model:

============================
Error Message:
============================

An error occurred while opening the model on the workspace database. Reason: ReadElementContentAs() methods cannot be called on an element that has child elements.

============================
Call Stack:
============================

 at Microsoft.AnalysisServices.VSHost.VSHostManager.PrepareSandbox(Boolean newProject, Boolean& isRefreshNeeded, Boolean& isImpersonationChanged, Boolean& saveRequired, List`1& truncatedTables, Boolean isRealTimeMode, Int32 clientCompatibilityLevel)
 at Microsoft.AnalysisServices.VSHost.Integration.EditorFactory.CreateEditorInstance(UInt32 grfCreateDoc, String pszMkDocument, String pszPhysicalView, IVsHierarchy pvHier, UInt32 itemid, IntPtr punkDocDataExisting, IntPtr& ppunkDocView, IntPtr& ppunkDocData, String& pbstrEditorCaption, Guid& pguidCmdUI, Int32& pgrfCDW)

============================

This problem is a bug in SSDT in that it does not properly parse annotations created by BIDS Helper, and it has been reported to Microsoft here. If you are experiencing this issue, please vote for it on Connect to raise its visibility. Until Microsoft fixes the problem, the BIDS Helper team has created a workaround utility which changes the storage format for BIDS Helper annotations. BIDS Helper release 1.6.2 includes this workaround feature.

If you are experiencing this ReadElementContentAs() error, then consider using this BIDS Helper workaround utility. With the .bim file closed, right click on the .bim file and select Tabular Annotation Workaround...

This feature will popup a confirmation dialog. Read it carefully, especially the note about every developer on your team upgrading to the latest version of BIDS Helper.

Updated Wiki: Documentation

furmangg — Wed, 27 Feb 2013 03:58:12 GMT

Installation

To install BIDS Helper, download the installer from the Releases tab.
If for some reason you cannot use the installer the latest release includes an xcopy deploy option.

Suggestions

We are always looking for potential new features, and we maintain a list of items in the Issue Tracker. If you see an item you would like addressed in the Issue Tracker please vote for it. If you have a feature you would like to see added, drop us a line on the discussion forum.

Features

Developers

For information about compiling the latest source code yourself, please see this discussion

New Comment on "Tabular Annotation Workaround"

Albuntu — Mon, 25 Feb 2013 12:15:46 GMT

As I also run into this issue today I was wondering where one can download this Workaround?

New Comment on "Create Fixed Width Columns"

vansree — Fri, 15 Feb 2013 00:34:51 GMT

Nice Feature! Is there a way to export the list back? Thanks Vankayala

New Comment on "xcopy deploy"

gbrueckl — Fri, 01 Feb 2013 17:38:15 GMT

for BIDS Helper 2012 I had to copy the DLL to C:\Program Files (x86)\BIDS Helper 2012\BIDShelper.dll

Updated Wiki: Tabular Annotation Workaround

furmangg — Mon, 21 Jan 2013 15:33:44 GMT

Tabular Annotation Workaround

Note: This feature is not in the latest release. You need release 1.6.2 to use this feature.

After upgrading to SQL Server 2012 SP1 SSDT, some users have complained about receiving an error that prevents them from opening the .bim file in a Tabular model:

============================
Error Message:
============================

An error occurred while opening the model on the workspace database. Reason: ReadElementContentAs() methods cannot be called on an element that has child elements.

============================
Call Stack:
============================

 at Microsoft.AnalysisServices.VSHost.VSHostManager.PrepareSandbox(Boolean newProject, Boolean& isRefreshNeeded, Boolean& isImpersonationChanged, Boolean& saveRequired, List`1& truncatedTables, Boolean isRealTimeMode, Int32 clientCompatibilityLevel)
 at Microsoft.AnalysisServices.VSHost.Integration.EditorFactory.CreateEditorInstance(UInt32 grfCreateDoc, String pszMkDocument, String pszPhysicalView, IVsHierarchy pvHier, UInt32 itemid, IntPtr punkDocDataExisting, IntPtr& ppunkDocView, IntPtr& ppunkDocData, String& pbstrEditorCaption, Guid& pguidCmdUI, Int32& pgrfCDW)

============================

This problem is a bug in SSDT in that it does not properly parse annotations created by BIDS Helper, and it has been reported to Microsoft here. If you are experiencing this issue, please vote for it on Connect to raise its visibility. Until Microsoft fixes the problem, the BIDS Helper team has created a workaround utility which changes the storage format for BIDS Helper annotations.

If you are experiencing this ReadElementContentAs() error, then consider using this BIDS Helper workaround utility. With the .bim file closed, right click on the .bim file and select Tabular Annotation Workaround...

This feature will popup a confirmation dialog. Read it carefully, especially the note about every developer on your team upgrading to the latest version of BIDS Helper.

Updated Wiki: Tabular Annotation Workaround

furmangg — Mon, 21 Jan 2013 12:44:32 GMT

Tabular Annotation Workaround

============================
Error Message:
============================

An error occurred while opening the model on the workspace database. Reason: ReadElementContentAs() methods cannot be called on an element that has child elements.

============================
Call Stack:
============================

 at Microsoft.AnalysisServices.VSHost.VSHostManager.PrepareSandbox(Boolean newProject, Boolean& isRefreshNeeded, Boolean& isImpersonationChanged, Boolean& saveRequired, List`1& truncatedTables, Boolean isRealTimeMode, Int32 clientCompatibilityLevel)
 at Microsoft.AnalysisServices.VSHost.Integration.EditorFactory.CreateEditorInstance(UInt32 grfCreateDoc, String pszMkDocument, String pszPhysicalView, IVsHierarchy pvHier, UInt32 itemid, IntPtr punkDocDataExisting, IntPtr& ppunkDocView, IntPtr& ppunkDocData, String& pbstrEditorCaption, Guid& pguidCmdUI, Int32& pgrfCDW)

============================

This problem is a bug in SSDT in that it does not properly parse annotations created by BIDS Helper, and it has been reported to Microsoft here. If you are experiencing this issue, please vote for it on Connect to raise it's visiblity. Until Microsoft fixes the problem, the BIDS Helper team has created a workaround utility which changes the storage format for BIDS Helper annotations.

If you are experiencing this ReadElementContentAs() error, then consider using this BIDS Helper workaround utility. With the .bim file closed, right click on the .bim file and select Tabular Annotation Workaround...

This feature will popup a confirmation dialog. Read it carefully, especially the note about every developer on your team upgrading to the latest version of BIDS Helper.