Category Archives: Data Access

Another Use Case for DbContext.Add in EFCore (and a DDD win)

Data Access, Uncategorized,

If you are like me and design your classes following Domain-Driven Design principals , you may find yourself with code like this for controlling how objects get added to collections in the root entity.

public class Samurai {

  public Samurai (string name) : this()
  {
     Name = name;
  }

  private Samurai ()
  {
    _quotes=new List<Quote>();
  }

  public int Id { get; private set; }
  public string Name { get; private set; }
  private readonly List _quotes = new List ()
  private IEnumerable Quotes => _quotes.ToList ();
  public void AddQuote (string quoteText) {
      var newQuote=new Quote(quoteText,Id);
      _quotes.Add (newQuote);
  }

I have a fully encapsulated collection of Quotes. The only way to add a new quote is through the AddQuote method. You can’t just call Samura.Quotes.Add(myquote).

Additionally, because I want to control how developers interact with my API, there is no DbSet for Quotes. You have to do all of your queries and updates via context.Samurais.

A big downside to this is that if I have a new quote and I know the ID of the samurai, I have to first query for the samurai and then use the AddQuote. That really bugs me. I just want to create a new quote, push in the Samurai’s ID value and save it. And that requires either raw SQL or a DbSet<Quote>. I don’t like either option. Raw SQL is a hack in this case and DbSet<Quote> will open my API up to potential misuse.

I was thinking about this problem while laying in bed this morning (admit it, that’s the first thing you do when you wake up, too, right?) and had an idea.

In EF Core, we can now add objects directly to the context without going through the DbSet. The context can figure out what DbSet the entity belongs to and apply the right info to the change tracker. I thought this was handy for being able to call

myContext.AddRange(personobjectA, accountobjectB, productObjectC);

Although I haven’t run into a good use case for leveraging that yet.

What occurred to me is that if DbContext.Add is  using  reflection, maybe EF Core can find a private DbSet.

So I added a private DbSet to my DbContext class:

private DbSet<Quote> Quotes { get; set; }
 And tried out this code (notice I’m using context.Add, not context.Quotes.Add):
static void AddQuoteToSamurai () 
{
  using (var context =newSamuraiContext ()) 
  {
    var quote=newQuote("Voila",1);
    context.Add(quote);
    context.SaveChanges();
  }
}
And it worked! But this isn’t complete yet. I’m breaking my rule of ensuring that only my aggregate root can manage quotes. So this is “dangerous” code from my DDD perspective. However, I was happy to know that EF Core would support this capability.
Currently, Samurai.AddQuote does not have any additional logic to be performed on the quote. What if I were to add in a “RemoveBadWords” rule before a quote can get added?
public void AddQuote (string quoteText) 
{
 Utilities.RemoveBadWords(quoteText);
 var newQuote=new Quote(quoteText,Id);
  _quotes.Add (newQuote);
}
 Now I have an important reason to use Samurai to do the deed. I can add a second, static AddQuote method that also takes an int. Because it’s static, it’s a pass through method.
public static Quote AddQuote(string quoteText,int samuraiId)
{
  Utilities.RemoveBadWords(quoteText); 
  var newQuote=newQuote(quoteText,samuraiId);
  return newQuote;
}

This works and now I don’t have to have an instance of Samurai to use it:

staticvoid AddQuoteToSamurai () 
{
  using (var context =newSamuraiContext ()) {
    context.Add(Samurai.AddQuote("static voila",1));
    context.SaveChanges();
}

One thing I was worried about was if I had an instance of Samurai and tried to use this to add a quote to a different samurai. That would break the aggregate root…it’s job is to manage its own quotes only. It shouldn’t know about other Samurais.

But .NET protects me from that. I can’t call the static method from an instance of Samurai.

I still think that there’s a little bit of code smell from a DDD perspective about having this static, pass-through method in an aggregate root so will have to investigate that (or wait for any unhappy DDDers in my comments). But for now I am happy that I can avoid having to query for an instance of Samurai just to do this one task.

First Foray into .NET Core 2.0

Data Access, , ,

I have to start somewhere so I started wtih super baby steps. Downloading the .NET Core 2 nightly build and trying to create a simple console app. Right away I failed miserably. The reason? The nightly builds have already jumped the shark! They are working on 2.1.0 and I accidentally grabbed that. And that was a little too bleeding edge for me.

There is a docker image that you can use quite easily but I wanted to try CLI, VS Code and Visual Studio. Therefore I wanted to just install the bits right on my machine.

What I’ll show you are  my first tests I did on macOS and on Windows 10. I like to do this just to make sure things are actually running properly. Note that on macOS, I just installed this new version directly on my machine where I have other versions of .NET Core. Key is that there is no production work on there dependent on other versions, so I won’t mess up anything important. The versions can live side by side. It’s just that for someone like me who “knows enough to be dangerous”, it’s easy to get tangled up with the versions even though I know tricks like creating a local nuget.config file and also specifying a version in global.json. On Windows, however, I am using a clean VM that has no other versions of .NET on it. That’s the smart way anyway. Although I did try the not smart way of installig it on my machine that already has all kinds of versions of all kinds of frameworks on it and even with my versioning tricks, I could not get 2.0.0 to do a restore or build.

I mentioned above that I originally downloaded the wrong version of the SDK. (Note that the typical SDK install also includes the runtime….so I got the wrong versoins of both. You can read the gory details of that in this GitHub issue which I kept updating as I sorted the problem out.) I want to stick with .NET Core 2.0.0. The installer for that is tucked away in a branch of github.com/dotnet/cli rather than grabbing the absolute latest from the master. Instead go to https://github.com/dotnet/cli/tree/release/2.0.0. There is a solid 2.0.0 version — 2.0.0-preview2-006391. That’s the one I’m using on Windows and macOS.

2017-06-11_12-23-16.jpg

Make Sure NuGet Knows Where to Find Packages

You can update your global NuGet.Config before creating projects. Alternatively, you can create a project and then add a local NuGet.config file to it.

On Mac, the global config file at [user]/.nuget/NuGet. Here s a screenshot if like me, macOS is not your primary platform and you still struggle with these things.

2017-06-11_12-32-15.jpg

In Windows, it’s at %APPDATA%\NuGet\.

I keep a link to the “Configuring NuGet behavior” doc handy for when I forget where to find that.

I added these two keys to my PackageSources section:

<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />

<add key="dotnet-core" value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json"/>

 Now it’s time to create a project

I’m on the Mac, so, to Terminal we will go.

I’ve created a new folder called EFCore20 – I know this is .NET Core, but eventually I plan to add in EF Core 2.0 as well.
My plan is to create a .NET Core console app (this will rely on netcoreapp2.0) and a library. The libray will be based on the netstandard2.0 library . That means its a library I’ll be able to use from a huge variety of apps, including .NET 4.6.1 based apps. EF Core 2.0 will also rely on .NET Standard 2.0, so any app or API that can consume netstandard, can also consume EF Core 2.0. Read more about that in this EF Core 2.0 announcement on GitHub.

I created one folder for each inside the EFCore20 folder:

netcore2console
netstandard2lib

Then I cd’d into the netcore2console app to create that app with the command:

dotnet new console.
That’s all it takes. This creates a tiny little Hello World app. The dotnet new command will also perform a dotnet restore after creating the files. This is the first moment you will see if things are wokring again. If the restore fails, it will tell you immediately.
This is the message I got when had things misaligned:
Error message: error MSB4236: The SDK ‘Microsoft.NET.Sdk’ specified could not be found.
The “specified” SDK is the currently installed version of whatever SDK you are referencing. In my case, the default for dotnet enw console is “dotnetcore” with the version being whatever version is installed.
Most likely the problem is that  you haven’t provided the correct URI for dotnet to find the proper NuGet packages.
If you don’t want to mess with the global config, you can create a local NuGet.config file in the folder with the app. In it’s entirety, it would look like:
<?xml version="1.0" encoding="utf-8"?>
 <configuration>
  <packageSources>
   <addkey="nuget.org"value="https://api.nuget.org/v3/index.json"protocolVersion="3"/>
   <addkey="dotnet-core"value="https://dotnet.myget.org/F/dotnet-core/api/v3/index.json"/>
  </packageSources>
</configuration>
Even though dotnet new will call restore, I like to call
dotnet restore
explicitly. (Control freak)
If it restored correctly, then the next step for validation is
dotnet build
And hopefully that gives you no errors as well. It shouldn’t.
Finally,
dotnet run
should result in spitting out
Hello World!
And now I know it’s working. Silly little bit but I want to verify this before I waste my time writing a bunh of code that isn’t going to run because I don’t even have .NET Core installed properly.
Another point of interest is what files were created by dotnet new console.
It’s easier to see them if I open them up in Visual Studio Code which I can do by typing
code .
There are only 2 files. The csproj with the project metadata and a program file.  The csproj file says that the target framework is netcoreapp2.0.

2017-06-11_14-09-46

The program file just spits out Hello World when it starts up.
2017-06-11_14-10-17
Here is a screenshot of all of the steps at the command line from dotnet new console to the Hello World! output including my extra dotnet restore. Another thing I like about the explicit restore is that it’s showing me more detailed info about the restore.
2017-06-11_14-03-19
Next I want to make sure I can get a .NET Standard library also working.
Still in the terminal (or console or PS if you’re on Windows), I now get into the 2nd sub folder, EF Core2/netstandardlibrary. dotnet new library defaults to using netstandard for the library. So that makes it easy.
dotnet new library
The library is created and the packages it needs are restored.
Here are the default contents of this new library, again, shown in VS Code.
Ntice that its csproj says the target framework is netstandard2.0.2017-06-11_14-16-09
And there’s an empty class file.
2017-06-11_14-18-12
I’ll modify the class to add a public method that returns a string: “Why, Hello There!”.
2017-06-11_14-22-01

Now I’ll go into the csproj for the console app and give it a project reference to the library. The intellisense does not (yet?) help me with this and my memory sucks*, so I head to Nate McMaster’s handy project.json to csproj mind mapper aka Project.json to MSBuild conversion guide to remind myself the syntax of a project reference. I add this below the property group section in the console app’s csproj file

 

<ItemGroup>
  <ProjectReference Include="..\netstandard2lib\netstandard2lib.csproj"/>
</ItemGroup>
Running dotnet restore and dotnet build again will help identify if you’ve got a typo in there. I often do.
Now I’ll modify the Program.cs file to also call output the resultls of caling the HiYa method from my library:
2017-06-11_14-31-39

And then back out to my terminal window (though I can also do it inside VS Code’s built-in terminal). I dotnet build the library and then change to the console folder. Rebuild that and run it and …voila

➜  netcore2console dotnet run

Hello World!

Why, Hello There!

➜  netcore2console

So now I know that .NET Core 2 (preview from a nightly build) is running properly on my machine and that I can create and use a .NET Standard 2.0 library. That means I can go ahead and confidently start creating a .NET Standard 2.0 library to host some EF Core 2.0 logic.

I also repeated this entire thing on my Windows machine. Not only does that let me know I can use this version of .NET Core there, but I will also do some work from within Visual Studio with the .NET Core 2.0 preview.

*(duh, I should just make myself a snippet in VS Code!)

What’s in that sqlservr.sh file on the mssql-sqlserver-linux docker image anyway?

Data Access, ,

Update June 3, 2017: The team has revised the docker image and the bash file is gone, presumably with its logic broken up in to various locations. Still I’m glad I grabbed this when I did to satisfy my curiosity!

Microsoft has created 4 official Docker images for SQL Server: SQL Server for Linux, SQL Server Developer Edition, SQL Server Express and (windows) SQL Server vNext) . They can be found on the Docker hub (e.g. https://hub.docker.com/r/microsoft/mssql-server-linux/) and there is also a Github repository for them at github.com/Microsoft/mssql-docker. Some of the files that go along with that image are not on Github. The Dockerfile files for each image run some type of startup script. The Windows images have a PowerShell script called start.ps1. You can see those in the Github repo. The Linux image runs a bash file called sqlservr.sh. That’s not included in the repo though and I was curious what it did.

Note: I wrote a blog post about using the SQL Server for Linux container (Mashup: SQL Server on Linux in Docker on a Mac with Visual Studio Code)  and I’m also writing an article about using the containers for my July MSDN Magazine Data Points column (watch this space).

Still a bit of a bash noob, I learned how to read a file from a docker container on ..you guessed it…StackOverflow.  Following those instructions, I created a snapshot of my running container

➜ MySqlServerLinuImage git:(master) docker commit juliesqllinux  mysnapshot

sha256:9b552a1e24df7652af0c6c265ae5e2d7cb7832586c431d4b480c30663ab713f0

and ran the snapshot with bash:


➜  MySqlServerLinuImage git:(master) docker run -t -i mysnapshot bin/bash

root@2a6f950face2:/#

Then at the new prompt (#), used ls to get the listing

root@2a6f950face2:/# ls

SqlCmdScript.sql  SqlCmdStartup.sh  bin  boot  dev  entrypoint.sh  etc  home  install.sh  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

then navigated to  folder where the bash file is and listed its contents:

root@2a6f950face2:/opt/mssql/bin# ls

compress-dump.sh  generate-core.sh  mssql-conf  paldumper  sqlpackage  sqlservr  sqlservr.sh

Once I was there I used the cat command to list out the contents of the sqlservr.sh file and see what it does. Here is the secret sauce in case, like me, you NEED to know what’s going on under the covers!

root@2a6f950face2:/opt/mssql/bin# cat sqlservr.sh 

#!/bin/bash

#

# Microsoft(R) SQL Server(R) launch script for Docker

#

ACCEPT_EULA=${ACCEPT_EULA:-}

SA_PASSWORD=${SA_PASSWORD:-}

#COLLATION=${COLLATION:-SQL_Latin1_General_CP1_CI_AS}

have_sa_password=""

#have_collation=""

sqlservr_setup_prefix=""

configure=""

reconfigure=""

# Check system memory

#

let system_memory="$(awk '/MemTotal/ {print $2}' /proc/meminfo) / 1024"

if [ $system_memory -lt 3250 ]; then

    echo "ERROR: This machine must have at least 3.25 gigabytes of memory to install Microsoft(R) SQL Server(R)."

    exit 1

fi

# Create system directories

#

mkdir -p /var/opt/mssql/data

mkdir -p /var/opt/mssql/etc

mkdir -p /var/opt/mssql/log

# Check the EULA

#

if [ "$ACCEPT_EULA" != "Y" ] && [ "$ACCEPT_EULA" != "y" ]; then

 echo "ERROR: You must accept the End User License Agreement before this container" > /dev/stderr

 echo "can start. The End User License Agreement can be found at " > /dev/stderr

 echo "http://go.microsoft.com/fwlink/?LinkId=746388." > /dev/stderr

 echo ""

 echo "Set the environment variable ACCEPT_EULA to 'Y' if you accept the agreement." > /dev/stderr

 exit 1

fi

# Configure SQL engine

#

if [ ! -f /var/opt/mssql/data/master.mdf ]; then

 configure=1

 if [ ! -z "$SA_PASSWORD" ] || [ -f /var/opt/mssql/etc/sa_password ]; then

 have_sa_password=1

 fi

# if [ ! -z "$COLLATION" ] || [ -f /var/opt/mssql/etc/collation ]; then

# have_collation=1

# fi

 if [ -z "$have_sa_password" ]; then

        echo "ERROR: The system administrator password is not configured. You can set the" > /dev/stderr

        echo "password via environment variable (SA_PASSWORD) or configuration file" > /dev/stderr

        echo "(/var/opt/mssql/etc/sa_password)." > /dev/stderr

 exit 1

 fi

fi

# If user wants to reconfigure, set reconfigure flag

#

if [ -f /var/opt/mssql/etc/reconfigure ]; then

 reconfigure=1

fi

# If we need to configure or reconfigure, run through configuration

# logic

#

if [ "$configure" == "1" ] || [ "$reconfigure" == "1" ]; then

 sqlservr_setup_options=""

# if [ -f /var/opt/mssql/etc/collation ]; then

# sqlservr_setup_options+="-q $(cat /var/opt/mssql/etc/collation)"

# else

# if [ ! -z "$COLLATION" ]; then

# sqlservr_setup_options+="-q $COLLATION "

# fi

# fi

 set +e

 cd /var/opt/mssql

 echo 'Configuring Microsoft(R) SQL Server(R)...'

 if [ -f /var/opt/mssql/etc/sa_password ]; then

 SQLSERVR_SA_PASSWORD_FILE=/var/opt/mssql/etc/sa_password /opt/mssql/bin/sqlservr --setup $sqlservr_setup_options 2>&1 > /var/opt/mssql/log/setup-$(date +%Y%m%d-%H%M%S).log

 elif [ ! -z "$SA_PASSWORD" ]; then

 SQLSERVR_SA_PASSWORD_FILE=<(echo -n "$SA_PASSWORD") /opt/mssql/bin/sqlservr --setup $sqlservr_setup_options 2>&1 > /var/opt/mssql/log/setup-$(date +%Y%m%d-%H%M%S).log

 else

 if [ ! -z '$sqlservr_setup_options' ]; then

 /opt/mssql/bin/sqlservr --setup $sqlservr_setup_options 2>&1 > /var/opt/mssql/log/setup-$(date +%Y%m%d-%H%M%S).log

 fi

 fi

 retcode=$?

 if [ $retcode != 0 ]; then

 echo "Microsoft(R) SQL Server(R) setup failed with error code $retcode. Please check the setup log in /var/opt/mssql/log for more information." > /dev/stderr

 exit 1

 fi

 set -e

 rm -f /var/opt/mssql/etc/reconfigure

 rm -f /var/opt/mssql/etc/sa_password

 echo "Configuration complete."

fi

# Start SQL Server

#

exec /opt/mssql/bin/sqlservr $*

Mashup: SQL Server on Linux in Docker on a Mac with Visual Studio Code

Data Access,

I’ve been having a lot of fun with the new mssql extension for Visual Studio Code. I have an article coming in MSDN Magazine and am planning more fun as well. My latest experiment was doing a big mashup taking advantage of the fact that there is now a Linux version of SQL Server. So we are no longer limited to hosting it on Windows or Azure. The most lightweight way to host SQL Server on Linux is in a Docker container. While I am sitting in front of a MacBook typing this I’m by no means working towards abandoning my Windows development or Windows machines. I’m just happy to have more options at my disposal as well as have the ability to share what I am learning work beyond the world of Windows developers.

Containers are not stateful. There are ways around that (I’ll show you below) but I only know enough to be dangerous here. This is a great way to use SQL Server at design time. Using this for production is a totally different story and you need to do a lot more research and soul-searching before using that option.

On the other hand, there are those who do have that particular goal:

I had to go through a number of documents to do this and of course I got stuck even with those resources at my disposal. So I will share the full path of how I got this setup working.

Pre-Requisite

I already have Docker for Mac installed on my MacBook. Here is the installation link if you need to perform that step. Keep in mind that you can’t do this on a VM. I tried as I wanted to repeat this with a clean setup.

Be sure that Docker is set to use at least 4GB of memory.

2017-04-07_13-55-27.jpg

Getting the SQL Server Docker Image

This is what makes the whole thing so easy! Microsoft has created an official docker image with SQL Server for Linux already on it.

In the terminal window, you can pull and install the official image with

 sudo docker pull microsoft/mssql-server-linux

Once it’s installed, the ‘docker images’ command will show you that the image is now available on your machine.

2017-04-07_14-10-55

Although I just installed it today, you can see that the image I’m using — which is the latest version — was created by Microsoft 3 weeks ago.

Spinning Up a Container (or Two) From the Image

Now that Docker is aware of the image, you can create a container from it — which is running instance of the image.

Because we’re on a Mac and awaiting for a “bug” to get fixed, we will actually create two containers.

Depending on your familiarity with Docker, you may or may not be aware that containers are not stateful. Once you delete a container, it’s all gone! If you have persisted data in that container, it, too, is all gone. However, Docker has a feature called “Volumes” which are a way to retain state between docker instances. So when one instance is shut down, the state is stored in a Volume. When another container instance is spun up, that volume provides the container with the state from the previous instance. This is how it’s possible to use containers for databases.

Here’s a great tutorial on volumes: https://rominirani.com/docker-tutorial-series-part-7-data-volumes-93073a1b5b72
And the official docker doc:  https://docs.docker.com/engine/tutorials/dockervolumes/#/creating-and-mounting-a-data-volume-container

However there’s an issue (which looks like the resolution is around the corner) with Docker on Mac hosting the sql-server-linux image. This prevents us from using a volume for persistence in the simple way. So instead, we’ll create a separate container that is a “data volume container”, then we will point the container that will run SQL Server to the data volume container.

Creating the volume container

I’ll name mine mssqldata. Here’s the command to create it. (Don’t miss the full length of the command!)

docker create -v /var/opt/mssql --name mssqldata  microsoft/mssql-server-linux /bin/true

This volume container still uses the image as its base. But we won’t be running SQL Server from this instance.

Creating the SQL Server container

Now you can create an container where you will run SQL Server and that container will use the data volume container for the persisted data.

docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=Passw0rd' -p 1433:1433 --volumes-from mssqldata -d --name sql-server microsoft/mssql-server-linux

The two environment variables (accept_eula and sa_password) are required. The userid is (gulp) ‘sa’. The password requirements are: “At least 8 characters including uppercase, lowercase letters, base-10 digits and/or non-alphanumeric symbols.”. Mine’s really fancy!

Once these exist,

docker ps

will only show you the regular container. The volume container is hidden so you need

docker ps -a

to see it.

2017-04-07_16-07-49.jpgNotice that the container I will run SQL Server on is on a port whereas the data volume container has a different status “Created”, and is not exposed on a port.

Test the Connection From the SQL Command Line

You don’t have to do this but it made me happy and it was fun. Did you know that you can interact with SQL Server from the command line? The tools are installed inside the container but that’s a messy way to use them. It’s easier to just install them on your computer directly. The command lines tools for Mac are sql-cli.

Note: On May 16th, Microsoft released the macOS version of sqlcmd and bcp (bulk copy). Good to know if you are already familiar with sqlcmd. Here’s their blog post with more details.

You can install that with:

npm install -g sql-cli

Start it up with the mssql command.

To minimum you need connect is to identify the server (which is at localhost by default, you don’t need to specify the port) and the password. It will presume sa for userid.

mssql -s localhost -p Passw0rd

If it’s successful, you’ll get some information followed by a new prompt, mssql.

Connecting to localhost...done

sql-cli version 0.4.14
Enter ".help" for usage hints.
mssql>

 Enough of that. Now I get to use my new favorite tool. The mssql extension in Visual Studio Code.

Connect in Visual Studio Code

Once you have mssql installed in VS Code, you can begin by creating a new sql file. Either via the commands (F1 for the command pallete, MS SQL to see the commands and then New Query. This will prompt you for a connection – one parameter at a time. YOu can also start with the MS SQL Connect command (⇧⌘C). The extension will prompt you for each parameter.

Server name:  localhost
Database name: (enter)
User name: sa
Password: Passw0rd
Save Password (yes or no)
Profile name: [your choice]

If you enter everything correctly, not only will it connect, but the details will get stored as a connection profile in the VS Code settings and be available for subsequent connections.

You can see the connection status as it is connecting and then when connected in the lower right hand corner of the IDE.

2017-04-07_18-19-37.jpg

In the SQL file open in the editor, you can type SQL and see some existing snippets as well as get help from Intellisense whicih has read the schema of the data on the server. So far there’s not much.

Selecting the  sqlListDatabases snippet and then executing it (right click for Execute Query on the context menu or ⇧⌘E) displays the databases:

2017-04-07_18-22-45.jpg

Now you can use TSQL to create databases, database objects, query data. In the results pane you will see data as a grid similar to what you might see in SSMS. You can also export results to CSV or JSON. I’ve recently written an article about all of the cool things you can do with mssql which will be in teh June 2017 MSDN Magazine. But that connects to a SQL Azure database. In the meantime you can just go to the docs for the extension (aka.ms/mssql-marketplace).

Creating a Database and a Table

I let some more snippets help me to create a database and a table.

The first was the sqlCreateDatabase where I changed the snippet’s database name placholder to create a new database called LinuxReally then executed that with ⇧⌘E.

Re-running the select name from sys.databases command showed that the new database was now in the list.

Next I leveraged the sqlCreateTable snippet to help me create a new table. I named teh table DatabasesIKnow and gave it three columns

CREATE TABLE dbo.DatabasesIKnow
(
  Id INT NOT NULL PRIMARY KEY, -- primary key column
  DatabaseName [NVARCHAR](50) NOT NULL,
  KnowIt [Bit]
);

For some reason, the intellisense cache did not automatically refresh when I created this. Possibly becuse it was a new database. Even the mssql extension’s Refresh Intellisense Cache command did not kick it in. I got it working by disconnecting and reconnecting this time choosing the LinuxReally database rather than letting the extension connect to master by default. When I did that, I could see the message “Updating Intellisense…” in the status bar. After I had done this, the intellisense did auto refresh any time I modified the database schema.

Once i had the new database, I could execute “select * from dbo.DatabasesIKnow” and see the proper results. In my case, since I haven’t added data, there were no rows. But clearly it was reading from my table.

2017-04-08_09-04-55.jpg

Taking Down the sql-server Container

Now come the big docker volume tests. I first disconnected from the database inside of VS Code with⇧⌘E. Then I stopped and removed the sql-server container with the two commands:

docker stop sql-server
docker rm sql-server

But I left the data volume container (mssqldata) running.

I then created a new container instance using the same command as earlier:

docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=Passw0rd' -p 1433:1433 --volumes-from mssqldata -d --name sql-server microsoft/mssql-server-linux

In VS Code, I think reconnect to the server and my new database which was easy since it had been stored in the connection profiles . It’s the first one, localhost: LinuxReally.

2017-04-08_09-11-50.jpg

The connection was successful and listing databases showed my new DatabasesIKnow database . Select * from that database showed the schema. So my database was persisted in the data volume container even though I had killed and recreated the sql server container.

Next test: take down both containers.

Now it was time to see what happens when I stop and remove both containers!

docker stop sql-server
docker rm sql-server 
docker stop mssqldata
docker rm mssqldata

Next I use the same command I used previously to restart the sql-server container with the parameter to use the (not running) mssqldata volume container.

docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=Passw0rd' -p 1433:1433 --volumes-from mssqldata -d --name sql-server microsoft/mssql-server-linux

That fails. I need to first run the data volume container. Unfortunately, I re-ran the create command for that and overwrote the existing container, so all was lost. GOOD LESSON THERE! 🙂

So I started from scratch. In the brand new data volume container I recreated my database and table.

After more experiments, I realized that I had misunderstood the Docker documentation on volumes. You can create copies of data volume containers and remove those. But you need to be more of a docker ninja. While you can make all kinds of copies of the container, once you RM all of the related volumes, it is gone gone. This blog post does clarify some of the info I was still confused about wrt creating/backing up/restoring data volume containers: tricksofthetrades.net/2016/03/14/docker-data-volumes/.

So the bottom line is you need to leave the Data Volume Container running in some format (the original or some flavor of copy). (I’m still finding it hard to believe that it doesn’t somehow get stored as a file you can re-run so I will update this as soon as someone corrects me!)

Windows, Mac, Linux, Azure and Anwhere You Can Host a Docker Container

And if you think this is only somehting to do on OS X (because that’s where I’m doing it) .. no no no! Did you know there is now Docker for Windows? And that VS Code is cross platform? This is such a great way to quickly get SQL Server up and running in your development environment.

2017-04-08_10-49-59.jpg

And as I said on twitter, comparing the experience of pulling the docker image and spinning up a container to the experience of installing SQL Server  on windows is something like this:

Quick Start EF Core Videos on Channel 9

Data Access, , , ,

My Pluralsight course, Entity Framework Core: Getting Started, is a pretty thorough exploration for learning about EF Core. But at 5 hours, it may not be the VERY first thing you want to look at. So I’ve created some 12 minute videos on Channel 9 that will, in that very short time, walk you through building some tiny apps that use EF Core so you can get your hands dirty. While you won’t learn much in the way of ins & outs, they will walk you through:

  • creating a new app from scratch,
  • adding in a domain class
  • add in EF Core
  • creating an EF Core data model
  • creating a database from the model
  • creating and saving some data
  • reading, updating and modifying some data.

That’s a lot for 12 minutes, but I have done it in a way that you can just watch and learn, or follow along to build the apps yourself . For some of the code that is not related to the EF Core lesson, you can even copy that code from my GitHub account and paste it into your solution.

There are currently 2 videos.

EF Core in a Full .NET Application Using Visual Studio 2015
This one is designed to assure you that you can use EF Core in a full .NET application and using totally familiar tools. You’ll build a little .NET console app that uses EF Core to read and write some movie data.

EFCoreQS1Lerman_512 2017-03-22_18-21-35

EF Core in an ASP.NET Core Application Using Visual Studio 2017
This video will give you the full leading edge .NET Core experience as you use the brand new Visual Studio 2017 to create a tiny ASP.NET Core MVC App that uses EF Core to interact with the data. Again, some of the code that is not about EF Core is available to copy/paste from my GitHub account, making it easy to follow along with the demo.EFCoreQS2Lerman_512

2017-03-22_18-19-44

I’ll be adding anther one showing  Creating an ASP.NET Core App with EF Core Using Visual Studio Code in OS X . You can keep an eye on my Channel 9 Niners page or this blog to see what that gets published.

When your ready to really learn about EF Core, head over to Pluralsight for my Entity Framework Core: Getting Started Course. If you don’t have a subscription, contact me for a 30-day free trial code.

2017-03-22_18-35-49

Changes to EF Core With the RTM of VS2017 and Tools

Data Access, ,

When Visual Studio 2017 released today a few other things happened that are relevant to Entity Framework Core.

For more on EF Core, watch my EF Core: Getting Started course on Pluralsight.

EF Core Migrations Tools Release

First – something we were prepared for – the .NET Core SDK was also released. The last stable version was 1.0.0-preview2-1-003137. It’s now simply 1.0.0. Along with this, its dependent tooling, including EF Core Tools for PowerShell and dotnet were also released. As the .NET Core support evolved from project.json to msbuild, the EF Core tools split . We have been using 1.0.0-preview4 (for .NET and project.json) and 1.0.0-msbuild3 for msbuild/csproj support.

Now the tool packages are 1.1.0 (Tools) and 1.0.0 (Tools.DotNet)

For PowerShell support: Microsoft.EntityFrameworkCore.Tools 1.1.0
For dotnet CLI support: Microsoft.EntityFrameworkCore.Tools.DotNet 1.0.0

In Visual Studio 2015 (for full .NET projects) and Visual Studio 2017 (shown here, for full .NET or .NET Core projects), the Package Manager will show the RTM versions:

image

Notice that I do not have “Include prerelease” checked.

If using PMC to install, it’s just

install-package Microsoft.EntityFrameworkCore.Tools

That’s for the PowerShell tools, otherwise, add .DotNet to the name.

But notice that you no longer need to add the –pre.

When using the CLI version of the tools, the command

dotnet ef –version

results in

Entity Framework Core .NET Command Line Tools 1.0.0-rtm-10308

Changes to Migrations Commands

As the tools evolved through the previews, some details changed for example, the scaffolding command got smarter.

But one change that is notable is with respect to EF Core in class libraries. You still need to point to an executable project (exe or test) to run most of the commands, but now you can at least just use “dotnet ef” to get the help file without having to set the –startup-project parameter. There are a few other commands that will run without knowledge of the startup project. You can read more about this in this GitHub thread. Check some of the later comments by Brice Lambson as he worked on evolving the commands.

EF Core 1.1.1 – Patch

This was a more subtle part of the release. Even though the 1.1.1 milestone on GitHub had 30 bug fixes that are all closed , there hadn’t been any mention that this was going to get pushed out and the milestone had no target date on it. Though I had my suspicions! Here’s a screenshot I happened to take on March 5.

image

And yes, the newest version of the EF Core packages is now 1.1.1. These are bug fixes …as the increment suggests.  Most of them are edge cases, but regardless, you should definitely update your EF Core packages to ensure you have these latest fixes. If you’re creating new projects, 1.1.1 is what you’ll see available from NuGet.

Note: there was a regression introduced in EF Core 1.1.1 that is targeted to get fixed with the next patch. You can read about this issue here: http://stackoverflow.com/questions/42708522/loading-related-data-aspnet-core-1-1

You can learn much more about EF Core in my EF Core: Getting Started course on Pluralsight.

Troubleshooting the dotnet ef command for EF Core Migrations

Data Access, ,

Updated March 7, 2017 after Visual Studio 2017 was released.
Also, keep in mind that I have been updating this post (and will continue to do so) as I discover new ways people are hitting problems with dotnet ef.

When using EF Core in a .NET Core app (ASP.NET Core or other app sitting on .NET core), it’s easy to run into a problem when attempting to use EF Core migrations at the command line. The most common one is

No executable found matching command "dotnet-ef"

A Note If Your Coming from My Pluralsight EF Core Course

Microsoft supports developing .NET Core apps in VS2017. The tooling for VS2015 is outdated and there are no plans to bring them up-to-date for the new csproj support. I recorded my Entity Framework Core: Getting Started  course on Pluralsight while VS2017 was still in beta. While I did recreate the VS2017 demos in RC3 right before we published the course, we chose to leave the  rest of  the .NET Core demos that are in VS2015 alone.  VS2015 only supports project.json and the project templates set you up for .NET Core 1.0, not .NET Core 1.1. So in the course means that we’re stuck with project.json support and tooling that’s not quite aligned. But Pluralsight and I both agreed that it made sense not to ALSO force users to the bleeding edge, not even released VS2017 for the demos. The course’s focus is on EF Core, so as long as I could hand-hold users through the project.json setup stuff without the need to make them expert at that, it was the right way to go.

Of the nearly 2000 who have already watched the course since it’s release less than 2 weeks ago,  a few people ran into some confusion with the versioning and getting the “no executable found” message. I worked through these with them but wanted to write down the suggestions I’d made and have a single blog post I could point to.

Problems You May Encounter with ‘dotnet ef’

While some of these notes are specific to the project.json use in the course, I’ve also added tips for using dotnet ef with the newer csproj/msbuild support.

There are a few key things to watch out for.

  1. The current stable tooling for EF Core migrations is split into two packages.
         Microsoft.EntityFrameworkCore.Tools is for PowerShell
        Microsoft.EntityFrameworkCore.Tools.DotNet is for the CLI (dotnet commands)
    Be sure you’ve referenced the Tools.DotNet version of the package so that you have access to the CLI commands. If you’re following my course, that’s explained.
  2. dotnet ef only works in .NET Core projects. If your project targets the full .NET framework, then you’ll need to use the PowerShell commands e.g. add-migration, update-database.
  3. Make sure that you are running the command from the folder that contains the project where the Tools package is referenced. This is explained in the course demos, but still a step you may overlook in your excitement!
  4. If you are using project.json*, make sure that you have the Tools.DotNet package in the Tools section, not the dependencies section. After March 7 release, this will just be “1.0.0”.
    *Going forward, you should only be using project.json with .NET Core 1.0 projects. If you are using the current .NET Core (1.1+) you should be using csproj/msbuild.

    "tools": {
   "Microsoft.EntityFrameworkCore.Tools.DotNet": "1.0.0"
 }
  5. If you’re using csproj/msbuild, make sure the tools package is listed in the DotNetCliToolsReference tag.  After March 7 release, this will just be version “1.0.0”.

    <DotNetCliToolReference   
    Include="Microsoft.EntityFrameworkCore.Tools.DotNet"
    Version=”1.0.0” />
  6. If you’re using csproj/msbuild, make sure the casing is correct if you’re manually adding that package in csproj! I’ve seen people get tripped up by typing Dotnet rather than DotNet.
  7. It’s possible that you have to Tools correctly placed but your IDE did not trigger a dotnet restore. So you may need to do that manually. Here’s an example where that bit someone. https://github.com/aspnet/EntityFramework/issues/7801
  8. You shouldn’t be using an RC of VS2017 at this point but I’m leaving this one here.
    If you are using an older Release Candidate of Visual Studio 2017 (before RC3) , the CLI tooling was not yet aligned with the msbuild support so with that version, you have to use the PowerShell commands to do migrations. Therefore you have to use the Tools package and work in the package manager console, not the command line.

    <DotNetCliToolReference   
    Include="Microsoft.EntityFrameworkCore.Tools"
    Version="1.1.0-preview4" />

    As of  VS2017 RC3 (which is what I show in the last module of the course) it was possible to use msbuild3 as shown in point #4 above and the CLI commands. After March 7 release, this will just be “1.0.0”.

  9. Be sure you’re targeting a relational database. Migrations only work with those and not, for example, InMemory. A twitter friend accidentally ran into this problem and was getting the “no executable found” error message.
  10. Make sure that the version of the EF Core tools you are using aligns with the version of .NET Core on your machine.

In my EF Core course, I’m using EF Core 1.1 and for EF tools and in all but the last module, I’m using Microsoft.EntityFrameworkCore.Tools.DotNet 1.0.0-preview4. (I’m in the process of  updating the course to use the new 1.0.0 package)

You’ll need .NET Core 1.1 installed and the related dotnet SDK – which is not numbered as simply. Today the  .NET Core SDK tools are still in preview so the version number of the current “stable” build that goes with .NET Core 1.1 is 1.0.0-preview2-1-001377.  

Note that on March 7  when VS2017 has its official release, the .NET Core SDK tools will also be released so the versions will just be normal numbers like 1.0.0.

Here’s an example of what the dotnet ef command will tell you if you don’t have .NET Core 1.1 installed:

The specified framework 'Microsoft.NETCore.App', version '1.1.0' was not found.
 - Check application dependencies and target a framework version installed at:
 C:\Program Files\dotnet\shared\Microsoft.NETCore.App
 - The following versions are installed:
 1.0.1
 - Alternatively, install the framework version '1.1.0'.

You can get the correct version via the download grid at microsoft.com/net/download/core. The grid only exposes stable releases. If you’re looking for nightly builds (which at this time you need for using the csproj support), there’s a link to those just below the grid.

The set of downloads you get via the LTS button is for .NET Core 1.0.3. The Current button gives you the latest stable versions. The SDK button gets you the Runtime + SDK, whereas the Runtime button gives you ONLY the runtime.

The SDK installs will give you the SDK and the runtime. When you’ve selected the SDK set of installs, it says that it’s .NET Core 1.0. That’s referring to the version of the SDK. It will also install both the .NET Core 1.0 and .NET Core 1.1 runtimes. That single SDK is able to work with both of the runtimes. I’m on Windows x64 so my download is the first one on the list…the Windows x64 installer.netcoresdk10

Just as an FYI, if you select the Runtime set of downloads, then you will only be getting a specific version of a runtime and not the SDK.

netcore11runtime

After it’s installed, typing dotnet will show you that you the runtime version that your machine is running by default. That’s the later one. After installing the 1.0 SDK with both runtimes, dotnet tells me I’m running Version 1.1.0 of the runtime. 

 dotnet –version gives you the version of the SDK. That’s already showing me that there was a patch because the result says “1.0.1”.

If you installed this new SDK but are still seeing the old SDK version (1.0.0-preview2-001313), that is likely because that version is specified in the global.json file of your solution. That shouldn’t create a problem for using the migration commands, but it’s a good idea to have the correct version listed in global.json.

Some additional tips for you!

Commands can only run from an executable/test project

In my Pluralsight EFCore course, I have the DbContext in its own class library project. So when running dotnet ef, after solving all of the above problems, you’ll get a new message which is just pointing out that the commands depend on an executable to run. That message looks like this:

Could not invoke this command on the startup project 'TestEFCore.Data'. This version of the Entity Framework Core .NET Command Line Tools does not support commands on class library projects in ASP.NET Core and .NET Core applications.

In my case, I wasn’t ready to add a UI or test to the solution,  so I added a minimal console app just to cover this need. And it needs to reference the project with the DbContext. Once that’s sorted, you can use the —startup-project parameter of the dotnet ef command to point to that project. While I show all of this in my course, you can also see that in my MSDN Magazine article here: msdn.microsoft.com/magazine/mt742867

This will get a touch easier with the msbuild version of the EF Core tools. With the newer tooling, you’ll at least be able to run dotnet ef to get the command’s help without pointing to a startup-project, although to run sub commands, you’ll still need to specify the startup project.

Installing the EF Core Tools via NuGet in Visual Studio 2015

One of the viewers of the course reported a strange problem. He was able to add the EF Core Tools package in project.json and use the migrations from the CLI as expected. But if he attempted to add the package from the NuGet Package Manager or the Package Manager Console instead, he was back to the old ‘No executable found matching command “dotnet-ef”‘ error.

He finally noticed that there were errors when NuGet attempted to download the package. But in his IDE the errors were subtle, so he was unaware that the tools package was not installed. Here’s a screenshot he shared:

frsBKTFTRSjBGOFFZDJ6_nugetmanager

I figured out how to get myself into a similar bind and got a much more helpful error message:

2017-02-27_17-08-52

Package 'Microsoft.EntityFrameworkCore.Tools.DotNet 1.1.0-preview4-final' uses features that are not supported by the current version of NuGet. To upgrade NuGet, see http://docs.nuget.org/consume/installing-nuget.

Even though Visual Studio extension manager did not indicate an available update AND it listed my NuGet Package Manager version as 3.5 (the latest), I learned that the team responsible for this extension had temporarily stopped pushing notifications for updates! That change is noted in the “No Auto Updates” section of this blog post: http://blog.nuget.org/20161027/Announcing-NuGet-3.5-RTM.html

So I had to explicitly download the latest VSIX (ignoring the fact that it seemed to have the same version # as the one I had installed!) from the NuGet  distributions page. After installing that, it resolved the problem of installing the EF Core tools package from the Package Manager and Package Manager Console.

2017-02-27_17-14-46

There’s a sneak peek at EF Core with msbuild in VS2017

A few people mentioned to me that they decided to go straight to VS2017 when working through the demos in the course. And they also said they were confused by some differences and had to do some research. I know for sure that one of these devs was kicking himself when I asked if he had watched the VS2017 demo I did at the end of the course before trying to VS2017 along with the early demos. (His answer was “umm, now you tell me!”)  It’s listed in the table of contents for the course. So if you take a peek at that first, I think doing all the demos in VS2017 will be a lot easier! I am currently in the process of updating the VS2017 demos to use the RTM and latest tools.

Hope this helps!

Entity Framework Core: Getting Started on Pluralsight

Data Access, ,

I’m happy to share a new course on Pluralsight with you – Entity Framework Core: Getting Started.

Here’s how I described it in the trailer:

Most software – whether for business or entertainment – is driven by data that users need to interact with. In Entity Framework Core: Getting Started, you will learn how to use Microsoft’s modern data access platform, Entity Framework Core. You will learn how to build data models, use EF Core to bridge your software with  your data store and how to incorporate all of this into desktop, mobile and web applications. When you’re finished with this course, you will have a foundational knowledge of Entity Framework Core that will help you as you move forward to build software in .NET, whether you are targeting Windows, OS X or Linux. Software required: Visual Studio 2015 or Visual Studio 2017.

pluralsight course

2017-02-15_17-54-44

 

Here is the list of modules in the course. You can see the titles of various clips in each module on Pluralsight.

To see the full list of my courses on Pluralsight, go to pluralsight.com/authors/julie-lerman

EF Core CLI Commands with VS2017 RC3

Data Access, , ,

Visual Studio 2017 RC3 was released yesterday but unfortunately an install issue has take it back off the shelf for a brief period. Watch this space for the return of RC3!

But I did manage to get it installed and wanted to show you that the EF Core CLI commands are now working. If you’ve been playing with VS2017 RC and EF Core you may have run into the problem that the EF Core tooling package was not in sync yet with the MSBuild tooling for .NET Core. That’s fixed now and not only does it work but there’s a change that I’m really happy to see.

As always, I have my dbcontext in its own project. Here are the csproj contents for that project:

image

 

Notice that the DotNetCliToolReference is pointing to Microsoft.EntityFrameworkCore.Tools.DotNet . The dotnet and PowerShell commands are exposed in separate packages. With “.DotNet” is the package that has the CLI commands. Without “.DotNet” is the package that contains the PowerShell commands.

More importantly, the package version went from “1.1.0-preview4” to “1.0.0-msbuild3-final”. I can’t explain why we went from 1.1.0 down to 1.0.0 but this is the newer and correct package.

With that in place,  I then open up a command prompt. I can use a regular one but I’m using a PowerShell command for a single benefit…that I can shorten the prompt. Here’s the command I did to trim most but not all of the path:

Quora: How do I get just the current folder name in my Windows Powershell prompt function?

Remember that I’m pointed to the path of a class library. DotNet EF requires you to point to a path containing an executable in order to run the commands. However with the latest bits, you can get HELP without having to point to the executable. Thank you Brice Lambson. It was a little meta to have to figure that out because rather than just getting help from the command, you had to googlebing for help on how to get help. So here are a simple dotnet ef command to get top level dotnet ef help, followed by dotnet ef dbcontext to get help on the dbcontext sub-commands.

image

To run commands that depend on the APIs, you still have to point to a startup-project if you are running the commands from a class library. Here I’ve run the command to list the migrations in my project. I’ve only got one, sqlite-init.

image