Upgrading my G/L Source Names Extension to AL – step 3

When upgrading an extension from C/AL to AL (version 1 to version 2) we need to think about the data upgrade process.

In C/AL we needed to add two function to an extension Codeunit to handle the installation and upgrade.  This I did with Codeunit 70009200.  One function to be execute once for each install.

PROCEDURE OnNavAppUpgradePerDatabase@1();
VAR
  AccessControl@70009200 : Record 2000000053;
BEGIN
  WITH AccessControl DO BEGIN
    SETFILTER("Role ID",'%1|%2','SUPER','SECURITY');
    IF FINDSET THEN REPEAT
      AddUserAccess("User Security ID",PermissionSetToUserGLSourceNames);
      AddUserAccess("User Security ID",PermissionSetToUpdateGLSourceNames);
      AddUserAccess("User Security ID",PermissionSetToSetupGLSourceNames);
    UNTIL NEXT = 0;
  END;
END;

And another function to be executed once for each company in the install database.

PROCEDURE OnNavAppUpgradePerCompany@2();
VAR
  GLSourceNameMgt@70009200 : Codeunit 70009201;
BEGIN
  NAVAPP.RESTOREARCHIVEDATA(DATABASE::"G/L Source Name Setup");
  NAVAPP.RESTOREARCHIVEDATA(DATABASE::"G/L Source Name User Setup");
  NAVAPP.DELETEARCHIVEDATA(DATABASE::"G/L Source Name");

  NAVAPP.DELETEARCHIVEDATA(DATABASE::"G/L Source Name Help Resource");
  NAVAPP.DELETEARCHIVEDATA(DATABASE::"G/L Source Name User Access");
  NAVAPP.DELETEARCHIVEDATA(DATABASE::"G/L Source Name Group Access");

  GLSourceNameMgt.PopulateSourceTable;
  RemoveAssistedSetup;
END;

For each database I add my permission sets to the installation users and for each company I restore the setup data for my extension and populate the lookup table for G/L Source Name.

The methods for install and upgrade have changed in AL for extensions version 2.  Look at the AL documentation from Microsoft for details.

In version 2 I remove these two obsolete function from my application management Codeunit and need to add two new Codeunits, one for install and another for upgrade.

codeunit 70009207 "O4N GL Source Name Install"
{
    Subtype = Install;
    trigger OnRun();
    begin
    end;

    var
    PermissionSetToSetupGLSourceNames : TextConst ENU='G/L-SOURCE NAMES, S';
    PermissionSetToUpdateGLSourceNames : TextConst ENU='G/L-SOURCE NAMES, E';
    PermissionSetToUserGLSourceNames : TextConst ENU='G/L-SOURCE NAMES';

    
    trigger OnInstallAppPerCompany();
    var
        GLSourceNameMgt : Codeunit "O4N GL SN Mgt";
    begin
        GLSourceNameMgt.PopulateSourceTable;
        RemoveAssistedSetup;
    end;

    trigger OnInstallAppPerDatabase();
    var
        AccessControl : Record "Access Control";
    begin
        with AccessControl do begin
            SETFILTER("Role ID",'%1|%2','SUPER','SECURITY');
            if FINDSET then repeat
                AddUserAccess("User Security ID",PermissionSetToUserGLSourceNames);
                AddUserAccess("User Security ID",PermissionSetToUpdateGLSourceNames);
                AddUserAccess("User Security ID",PermissionSetToSetupGLSourceNames);
            until NEXT = 0;
        end;
    end;

  local procedure RemoveAssistedSetup();
  var
    AssistedSetup : Record "Assisted Setup";
  begin
    with AssistedSetup do begin
      SETRANGE("Page ID",PAGE::"O4N GL SN Setup Wizard");
      if not ISEMPTY then
        DELETEALL;
    end;
  end;

  local procedure AddUserAccess(AssignToUser : Guid;PermissionSet : Code[20]);
  var
    AccessControl : Record "Access Control";
    AppMgt : Codeunit "O4N GL SN App Mgt.";
    AppGuid : Guid;
  begin
    EVALUATE(AppGuid,AppMgt.GetAppId);
    with AccessControl do begin
      INIT;
      "User Security ID" := AssignToUser;
      "App ID" := AppGuid;
      Scope := Scope::Tenant;
      "Role ID" := PermissionSet;
      if not FIND then
        INSERT(true);
    end;
  end;

}

In the code you can see that this Codeunit is of Subtype=Install.  This code will  be executed when installing this extension in a database.

To confirm this I can see that I have the G/L Source Names Permission Sets in the Access Control table .

And my G/L Source Name table also has all required entries.

Uninstalling the extension will not remove this data.  Therefore you need to make sure that the install code is structured in a way that it will work even when reinstalling.  Look at the examples from Microsoft to get a better understanding.

Back to my C/AL extension.  When uninstalling that one the data is moved to archive tables.

Archive tables are handled with the NAVAPP.* commands.  The OnNavAppUpgradePerCompany command here on top handled these archive tables when reinstalling or upgrading.

Basically, since I am keeping the same table structure I can use the same set of commands for my upgrade Codeunit.

codeunit 70009208 "O4N GL SN Upgrade"
{
    Subtype=Upgrade;
    trigger OnRun()
    begin
        
    end;
    
    trigger OnCheckPreconditionsPerCompany()
    begin

    end;

    trigger OnCheckPreconditionsPerDatabase()
    begin

    end;
    
    trigger OnUpgradePerCompany()
    var
        GLSourceNameMgt : Codeunit "O4N GL SN Mgt";
        archivedVersion : Text;
    begin
        archivedVersion := NAVAPP.GetArchiveVersion();
        if archivedVersion = '1.0.0.1' then begin
            NAVAPP.RESTOREARCHIVEDATA(DATABASE::"O4N GL SN Setup");
            NAVAPP.RESTOREARCHIVEDATA(DATABASE::"O4N GL SN User Setup");
            NAVAPP.DELETEARCHIVEDATA(DATABASE::"O4N GL SN");

            NAVAPP.DELETEARCHIVEDATA(DATABASE::"O4N GL SN Help Resource");
            NAVAPP.DELETEARCHIVEDATA(DATABASE::"O4N GL SN User Access");
            NAVAPP.DELETEARCHIVEDATA(DATABASE::"O4N GL SN Group Access");

            GLSourceNameMgt.PopulateSourceTable;
        end;
    end;

    trigger OnUpgradePerDatabase()
    begin

    end;

    trigger OnValidateUpgradePerCompany()
    begin

    end;

    trigger OnValidateUpgradePerDatabase()
    begin

    end;
    
}

So, time to test how and if this works.

I have my AL folder open in Visual Studio Code and I use the AdvaniaGIT command Build NAV Environment to get the new Docker container up and running.

Then I use Update launch.json with current branch information to update my launch.json server settings.

I like to use the NAV Container Helper from Microsoft  to manually work with the container.  I use a command from the AdvaniaGIT module to import the NAV Container Module.

The module uses the container name for most of the functions.  The container name can be found by listing the running Docker containers or by asking for the name that match the server used in launch.json.

I need my C/AL extension inside the container so I executed

Copy-FileToNavContainer -containerName jolly_bhabha -localPath C:\NAVManagementWorkFolder\Workspace\GIT\Kappi\NAV2017\Extension1\AppPackage.navx -containerPath c:\run

Then I open PowerShell inside the container

Enter-NavContainer -containerName jolly_bhabha

Import the NAV Administration Module

Welcome to the NAV Container PowerShell prompt

[50AA0018A87F]: PS C:\run> Import-Module 'C:\Program Files\Microsoft Dynamics NAV\110\Service\NavAdminTool.ps1'

Welcome to the Server Admin Tool Shell!

[50AA0018A87F]: PS C:\run>

and I am ready to play.  Install the C/AL extension

Publish-NAVApp -ServerInstance NAV -IdePath 'C:\Program Files (x86)\Microsoft Dynamics NAV\110\RoleTailored Client\finsql.exe' -Path C:\run\AppPackage.navx -SkipVerification

Now I am faced with the fact that I have opened PowerShell inside the container in my AdvaniaGIT terminal.  That means that my AdvaniaGIT commands will execute inside the container, but not on the host.

The simplest way to solve this is to open another instance of Visual Studio Code.  From there I can start the Web Client and complete the install and configuration of my C/AL extension.

I complete the Assisted Setup and do a round trip to G/L Entries to make sure that I have enough data in my tables to verify that the data upgrade is working.

I can verify this by looking into the SQL tables for my extension.  I use PowerShell to uninstall and unpublish my C/AL extension.

Uninstall-NAVApp -ServerInstance NAV -Name "G/L Source Names"
Unpublish-NAVApp -ServerInstance NAV -Name "G/L Source Names"

I can verify that in my SQL database I now have four AppData archive tables.

Pressing F5 in Visual Studio Code will now publish and install the AL extension, even if I have the terminal open inside the container.

The extension is published but can’t be installed because I had previously installed an older version of my extension.  Back in my container PowerShell I will follow the steps as described by Microsoft.

[50AA0018A87F]: PS C:\run> Sync-NAVApp -ServerInstance NAV -Name "G/L Source Names" -Version 2.0.0.0
WARNING: Cannot synchronize the extension G/L Source Names because it is already synchronized.
[50AA0018A87F]: PS C:\run> Start-NAVAppDataUpgrade -ServerInstance NAV -Name "G/L Source Names" -Version 2.0.0.0
[50AA0018A87F]: PS C:\run> Install-NAVApp -ServerInstance NAV -Tenant Default -Name "G/L Source Names"
WARNING: Cannot install extension G/L Source Names by Objects4NAV 2.0.0.0 for the tenant default because it is already installed.
[50AA0018A87F]: PS C:\run>

My AL extension is published and I have verified in my SQL server that all the data from the C/AL extension has been moved to the AL extension tables and all the archive tables have been removed.

Back in Visual Studio Code I can now use F5 to publish and install the extension again if I need to update, debug and test my extension.

Couple of more steps left that I will do shortly.  Happy coding…

 

Upgrading my G/L Source Names Extension to AL – step 2

So, where is step 1?  Step 1 was converting C/AL code to AL code.  This we did with AdvaniaGIT and was demonstrated here.

First thing first!  I received the following email from Microsoft.

Hello,

The decision has been made by our SLT, that the use of a Prefix or Suffix is now a mandatory requirement. If you are already using this in your app(s), great. If not, you will want to do so.

We are coming across too many collisions between apps in our internal tests during builds and have seen some in live tenants as well. It makes the most sense to make this a requirement now. If you think about it in a live situation, if a customer installs an app before yours and then tries yours but gets collision issues, they may just decide to not even continue on with yours.

Also, I have been made aware that adding a prefix or suffix after you already have a v2 app published can make the process complicated for you. Therefore, since you all have to convert to v2 anyway, now is a good time to add in the prefix/suffix.

The following link provides the guidelines around using it here

If you haven’t reserved your prefix yet, please email me back to reserve one (or more if needed).

Thank you,

Ryan

Since my brand is Objects4NAV.com I asked for 04N as my prefix and got it registered.  Since we got this information from Microsoft, every object that we develop in NAV 2018 now has our companies prefix in the name.

Starting my AL development by opening Visual Studio Code in my repository folder.  I updated my setup.json to match the latest preview build as Docker container and then selected to Build NAV Environment using AdvaniaGIT.

After download and deployment of the container I noticed that the container had a brand new version of the AL Extension for Visual Studio Code.  I looked at the version installed and that was an older version.

I uninstalled the AL Language extension and restarted Visual Studio Code.

As you can see on the screenshot above we now don’t have any AL Language extension installed.  I executed the Build NAV Environment command from AdvanaiGIT to install the extension on the Docker container.  In this case I already had a container assigned to my branch so only three things happened.

  • uidOffset in the container database was updated.  This is recommended for C/AL development.
  • License file is updated in the container database and container service.  The license used is the one configured in branch setup.json or the machine settings GITSettings.json
  • AL Language Extension is copied from the container to the host and installed in Visual Studio Code.

Again, restarting Visual Studio Code to find that the latest version of AL Language Extension has been installed.

I then executed two AdvaniaGIT actions.

  • Update Launch.json with current branch environment.  This will update the host name and the service name in my AL Launch.json file to make sure that my AL project will be interacting with the branch container.
  • Open Visual Studio Code in AL folder.  This will open another instance of Visual Studio Code in the AL folder.

Immediately after Visual Studio Code was opened it asked for symbols and I agreed that we should download them from the container.

Everything is now ready for AL development using the latest build that Microsoft has to offer.

I started  Edit – Replace in Files in Visual Studio Code.  All my objects have a name that start with G/L Source Name.  I used this knowledge to apply the prefix.

By starting with the double quote I make sure to only update the object names and not captions.  All captions start with a single quote.  I got a list of all changes and needed to confirm all changes.

The field name I add to G/L Entry table does not match this rule so I needed to rename that my self.  Selecting the field name and pressing F2 allows me to rename a field and have Visual Studio Code update all references automatically.

Pressing F5 started my build, publish and debug.

My extension is installed and ready for testing.

There are a few more steps that I need to look into before publishing the new version of G/L Source Names to Dynamics 365.  These steps will appear here in the coming days.  Hope this will be useful to you all.

Don’t worry about DotNet version in C/AL

When using DotNet data type in NAV C/AL we normally lookup a sub type to use.  When we do the result can be something like

Newtonsoft.Json.Linq.JObject.'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'

Then, what will happen when moving this code from NAV 2016 to NAV 2017 and NAV 2018.  The Newtonsoft.Json version is not the same and we will get a compile error!

Just remove the version information from the sub type information.

Newtonsoft.Json.Linq.JObject.'Newtonsoft.Json'

And NAV will find the matching Newtonsoft.Json library you have installed and use it.

This should work for all our DotNet variables.

AzureSQL database gives me change tracking error

I just uploaded a SQL bacpac to AzureSQL.  This I have done a number of times.  Connected my service to the SQL database and tried to start the service.

This time I got an error.  Looking in Event Viewer I can see.

The following SQL error was unexpected:
  Change tracking is already enabled for database '2018-IS365'.
  ALTER DATABASE statement failed.
  SQL statement:
  IF NOT EXISTS (SELECT * FROM sys.change_tracking_databases WHERE database_id=DB_ID('2018-IS365')) ALTER DATABASE [2018-IS365] SET CHANGE_TRACKING = ON (CHANGE_RETENTION = 10 MINUTES, AUTO_CLEANUP = ON)

I looked into the SQL database, and sure enough there was a line in sys.change_tracking_databases table.  The problem was that in that table the [database_id] was equal to 48 while

SELECT db_id('2018-IS365')

resulted in 49.  Hence the error and my service tier failing to start.

To remove the change tracking from the database I executed (found here)

DECLARE @SQL NVARCHAR(MAX)='';
SELECT @SQL = @SQL + 'ALTER TABLE ' + s.name + '.[' + t.name + ']' +
 ' Disable Change_tracking;'
FROM sys.change_tracking_tables ct
JOIN sys.tables t
 ON ct.object_id= t.object_id
JOIN sys.schemas s
 ON t.schema_id= s.schema_id;
PRINT @SQL;
EXEC sp_executesql @SQL;

ALTER DATABASE [2018-IS365] SET CHANGE_TRACKING = OFF;

The service tier will take care of turning the change tracking on again when it starts.  You might need to repeat these steps if restarting the service tier.

According to Microsoft a fix is in the pipeline and likely ships in CU2.

Using AdvaniaGIT – Convert G/L Source Names to AL

Here we go.

The NAV on Docker environment we just created can be used for the task at hand.  I have an Extension in Dynamics 365 called G/L Source Names.

I need to update this Extension to V2.0 using AL.  In this video I go through the upgrade and conversion process using AdvainaGIT and Visual Studio Code.

In the first part I copy the deltas from my Dynamics 365 Extension into my work space and I download and prepare the latest release of NAV 2018 Docker Container.

Using our source and modified environments we can build new syntax objects and new syntax deltas. These new syntax deltas are then converted to AL code.

 

Using AdvaniaGIT in Visual Studio Code

It has become obvious that the future of AL programming is in Visual Studio Code.

Microsoft has made a decision to ship all their releases as Docker Containers.

The result of this is a development machine that does not have any NAV version installed.  I wanted to go through the installation and configuration of a new NAV on Docker development machine.

Here is what I did.

I installed Windows Server 2016 with Containers.  The other option was to use Windows 10 and install Docker as explained here.

After installing and fully updating the operating system I downloaded and installed Visual Studo Code.

After installation Visual Studio Code detects that I need to install Git.

I selected Download Git and was taken to the Git download page.

I downloaded and installed Git with default settings.

To be able to run NAV Development and NAV Client I need to install prerequisite components.  I copied the Prerequisite Components folder from my NAV 2018 DVD and installed some of them…

Let’s hook Visual Studio Code to our NAV 2018 repository and install AdvaniaGIT.  I first make sure to always run Visual Studio Code with administrative privileges.

Now that we have our AdvaniaGIT installed and configured we can start our development.  Let’s start our C/AL classic development.  Where this video ends you can continue development as described in my previous posts on AdvaniaGIT.  AdvaniaGIT also supports NAV 2016 and NAV 2017.

Since we are running NAV 2018 we can and should be using AL language and the Extension 2.0 model.  Let’s see how to use our repository structure, our already build Docker container and Visual Studio Code to start our first AL project.

So as you can see by watching these short videos it is easy to start developing both in C/AL and AL using AdvaniaGIT and Visual Studio Code.

My next task is to update my G/L Source Names extension to V2.  I will be using these tools for the job.  More to come soon…

Using AdvaniaGIT – FTP server for teams

So, you are not the only one in your company doing development, right?

Essential part of being able to develop C/AL is to have a starting point.  That starting point is usually where you left of last time you did some development.  If you are starting a task your starting point may just be the localized release from Microsoft.

A starting point in AdvaniaGIT is a database backup.  The database backup can contain data and it should.  Data to make sure that you as a developer can do some basic testing of the solution you are creating.

AdvaniaGIT has a dedicated folder (C:\AdvaniaGIT\Backup) for the database backups.  That is where you should put your backups.

If you are working in teams, and even if not you might not want to flood your local drive with database backups.  That is why we configure an FTP server in C:\AdvaniaGIT\Data\GITSetting.json.

{
    "ftpServer":  "ftp://ftp02.hysing.is/",
    "ftpUser":  "ftp_sourcetree",
    "ftpPass":  "*****",
    "licenseFile":  "Advania.flf",
    "workFolder":  "C:\\NAVManagementWorkFolder\\Workspace",
    "patchNoFunction":  "Display-PatchDayNo",
    "defaultDatabaseServer":  "localhost",
    "defaultDatabaseInstance":  "",
    "objectsNotToDelete":  "(14125500..14125600)",
    "sigToolExecutable":  "C:\\Program Files (x86)\\Windows Kits\\10\\bin\\x64\\signtool.exe",
    "codeSigningCertificate":  "CodeSign_Certificate.pfx",
    "codeSigningCertificatePassword":  "****",
    "setupPath":  "setup.json",
    "objectsPath":  "Objects",
    "deltasPath":  "Deltas",
    "reverseDeltasPath":  "ReverseDeltas",
    "extensionPath":  "Extension1",
    "imagesPath":  "Images",
    "screenshotsPath":  "ScreenShots",
    "permissionSetsPath":  "PermissionSets",
    "addinsPath":  "Addins",
    "languagePath":  "Languages",
    "tableDataPath":  "TableDatas",
    "customReportLayoutsPath":  "CustomReportLayouts",
    "webServicesPath":  "WebServices",
    "binaryPath":  "Binaries",
    "testObjectsPath":  "TestObjects",
    "datetimeCulture":  "is-IS",
    "NewSyntaxPrefix":  "NewSyntax",
    "targetPlatform":  "DynamicsNAV",
    "VSCodePath":  "AL"
}

When we start an action to build NAV development environment the AdvaniaGIT tools searches for a database backup.

The search is both on C:\AdvaniaGIT\Backup and also on the root of the FTP server.

Using the function Get-NAVBackupFilePath to locate the desired backup file it will search based on these patterns.

    $FilePatterns = @(
        "$($SetupParameters.navRelease)-$($SetupParameters.projectName).bak",
        "$($SetupParameters.navRelease)/$($SetupParameters.navVersion)/$($SetupParameters.projectName).bak",
        "$($SetupParameters.navRelease)/$($SetupParameters.projectName).bak",
        "$($SetupParameters.navRelease)-$($SetupParameters.navSolution).bak"
        "$($SetupParameters.navRelease)/$($SetupParameters.navVersion)/$($SetupParameters.navSolution).bak",
        "$($SetupParameters.navRelease)/$($SetupParameters.navSolution).bak")

The navRelease is the year (2016,2017,…).  The navVersion is the build (9.0.46045.0,9.0.46290.0,10.0.17972.0,…)

The projectName and navSolution parameters are defined in Setup.json (settings file) in every GIT branch.

{
  "branchId": "3ba8870a-0274-4162-8ea2-66e314bb3e34",
  "navSolution":  "IS",
  "storeAllObjects":  "true",
  "navVersion":  "9.0.48992.0",
  "projectName": "ADIS",
  "baseBranch": "IS",
  "uidOffset": "10000200",  
  "objectProperties": "true",
  "datetimeCulture":  "is-IS"
}

Combining these values we can see that the search will be done with these patterns.

    $FilePatterns = @(
        "2016-ADIS.bak",
        "2016/9.0.48992.0/ADIS.bak",
        "2016/ADIS.bak",
        "2016-IS.bak"
        "2016/9.0.48992.0/IS.bak",
        "2016/IS.bak")

And these file patterns are applied both to C:\AdvaniaGIT\Backup and to the FTP server root folder.  Here are screenshots from our FTP server.

Looking into the 2017 folder

And into one of the build folders

My local backup folder is simpler

This should give you some idea on where to store your SQL backup files.

 

 

Using AdvaniaGIT – Docker Containers

There is a new kid in town.  His name is Docker.

Microsoft is saying:

We are currently investigating how we can use Docker for deploying NAV. For test purposes we have created a Docker Container Image with the NAV Developer Preview, which you can try out.

Docker Containers is a technology where you, instead of virtualizing the entire machine, only virtualize the services and share resources from the host computer. Read more about it here: https://www.docker.com/what-docker

Read more about how to get started with Docker here: https://docs.docker.com/docker-for-windows/install/

So what does this mean?

We can install NAV environments as container both in Azure and on premise.  We can have multiple NAV versions to work with without having to install, so there is no conflict.  We can also get access to container images that are still in preview.

Note what Microsoft says, they are investigating.  The NAV Container Image service is not public yet.  You need authentication to get access.  This project has a place on GitHub.  To get access please contact Microsoft or send me a line and I will point you in the right direction.

The easiest way to get started is to try the NAV Developer Preview template on Azure,  http://aka.ms/navdeveloperpreview.  This will give you a full development environment for NAV Extension2.0 using VS Code.

It should be straight forward to install AdvaniaGIT on top of the NAV Developer Preview and start from there.  We can also start from Azure template called “Windows Server 2016 Datacenter – with Containers”.

The local option is to install Docker on our Windows laptop.  If you would like to use Docker on your laptop you need to change one setting after installation.  You need to switch to Windows containers.  Your laptop will most likely restart during installation and configuration of Docker so make sure to have your work saved.

If you are planning to run a Docker-Only environment you don’t need to install NAV.  Still there are prerequisite components that you must install.  These components can be found on the NAV DVD folder “Prerequisite Components”.  From the “Microsoft SQL Server” folder install sqlncli64.msi and ReportBuilder3.msi.  From the “Microsoft Visual C++ 2013” folder install vcredist_x64.exe.  From the “Microsoft Visual Studio 2010 Tools for Office Redist” install vstor_redist.exe.  From the “Microsoft Report Viewer” folder install both SQLSysClrTypes.msi and ReportViewer.msi.  You should now be ready for the next step.

So, let’s get started

In your C:\AdvaniaGIT\Data folder open DockerSettings.json

{
  "RepositoryPath": "navdocker.azurecr.io",
  "RepositoryUserName": "7cc3c660-fc3d-41c6-b7dd-dd260148fff7",
  "RepositoryPassword": "access_key_for_the_repository"
}

That’s all.  Your are now ready to use Docker with AdvaniaGIT.  Make sure to always have the latest version of AdvaniaGIT installed.

You can even use the “devpreview” build of NAV TENERIFE to do vNext development both in C/AL and AL.

Stay tuned, the AdvaniaGIT journey will continue…