1.6.3.0 BIML Project parameters are not created

Jun 18, 2013 at 3:43 AM
This could be a PEBKAC issue, but I can't seem to get a BIML file to add project parameters. It seems to do just fine creating Package parameters.

Given the following markdown, I would expect
  1. A Project level parameter named "FolderBase" to be created
  2. A package named POC to be created
  3. A package parameter named Subfolder to be created in POC
  4. A Variable in the POC package to be an expression bringing the Project and Package variable together.
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
    <PackageProjects>
        <PackageProject Name="ProjectParameterTest" ProtectionLevel="DontSaveSensitive">
            <Parameters>
                <!-- I think this is the project parameter-->
                <Parameter DataType="String" Name="FolderBase">C:\ssisdata</Parameter>
            </Parameters>
            <Packages>
                <Package IsEntryPoint="true" PackageName="POC"/>
            </Packages>
        </PackageProject>
    </PackageProjects>
    <Packages>
        <Package Name="POC" ConstraintMode="Linear">
            <!-- Package parameter-->
            <Parameters>
                <Parameter DataType="String" Name="Subfolder">POC</Parameter>
            </Parameters>
            <Variables>
                <Variable Name="FolderInput" DataType="String" EvaluateAsExpression="true">@[$Project::FolderBase] + "\\" + @[$Package::Subfolder]</Variable>
            </Variables>
        </Package>
    </Packages>
</Biml>
I don't think it's entirely me doing it wrong. If I were to take out the Parameter declaration in the PackageProject node, the build will fail validation reporting the FolderBase project parameter doesn't exist. I also had a friend who owns Mist indicate the ability create project parameter does exist in there so I think the capability is there, just we need to patch it into BIDS Helper.

I'm guessing it'd be a STEP 4 in the Expand method of BIDSHelper.SSIS.Biml.BimlExpandPlugin to add all the declarations into the Project.params file but I'm just guessing based on the pattern.

If someone can give me a firm shove in the proper direction on how to get the debugger pointed at this thing, I'd be happy to give it a go and create a patch

Thank you all for the many years of hard work that have gone into BIDS Helper.
Developer
Jun 18, 2013 at 10:33 AM
Mist is capable of generating params files, so it is something that the core engine can do. However, this is something that isn't hard-coded as a method you can call externally. The engine heavily leverages BimlScript transformers to perform its own code generation. This is a core feature of the engine as it allows end-users to the paid product to fundamentally reprogram code emission (even to target entirely different platforms), if they want to. To learn more about BimlScript transformers, check out http://bimlscript.com/Walkthrough/Details/68.

Due to a variety of factors, we have to ship a very slimmed down set of transformers. The transformer that performs params emission is specifically omitted. While it could potentially be added back in, the current implementation of the transformer for params has a dependency on some other things (primarily package projects) that we can't easily turn on, so we'd need to refactor that transformer to work around the dependency. Refactoring the transformer is something Varigence would have to do. We don't allow BIDSHelper to do that. It's an advanced feature, and something we need to hold back for the paid product.

Actually refactoring that transformer hasn't been a high priority for a few reasons:
  1. You can create your parameters by hand and reference them from your projects without any problem. Auto-generating a large number of parameters is not something we've seen much of a need for.
  2. Since params are all stored in one file, you get a bunch of additional scenarios that you have to deal with. Here is a list of several off the top of my head:
    • If there are in memory changes to the params that haven't been saved, we probably need to force a save of that file.
    • There may be some hand-written params. What do we do with those? Merge them? If so, it would be a huge usability problem to go and delete any params that you autogen-ed by mistake, because on the recompile, they'd look like manually entered params that should be merged.
    • If you do a strict overwrite, then you have nasty scenarios where you actually want manual params of where you compile only a subset of your biml files and get some of your params wiped out.
    • What if there are multiple package projects in the code? Which params do we emit and merge?
  3. There are other corners - none are insurmountable, but it's a fair bit of non-trivial work. TLDR: It's not just the params transformer refactoring. It's all this other stuff, too.
  4. Other features directly related to params, such as configurations, require editing of the project file, which is a much worse can of worms.
Is this something you have a specific scenario for? If so, we might have some other ideas for you in the short term.