All posts by Julie Lerman

EF Core, Postgres and the Camel-Cased Identity Tables

PostgreSQL doesn’t really like camel-case too much. I’m no expert but I know that at least.

I was doing yet another exploration of  EF Core in Visual Studio Code on my MacBook. Usually I use JetBrain’s DataGrip (database IDE) to check out the actual data. But this time I wanted to play with  the Visual Studio Code extension called “vscode-database” which lets you interact with MySQL and PostgreSQL right in the IDE.

There are a bunch of database extensions in fact:

image

I’ve already been using the mssql extension to muck about with SQL Server data inside of VS Code:

As I was using a default web app from the template, ASP.NET Identity was involved and the context to manage identity inherits from Microsoft.AspNetCore.Identity.EntityFrameworkCore.IdentityDbContext

And IdentityDbContext has fluent API code to explicitly map table names which are not the same as the entity’s they are mapping. And it makes those names all camel-cased.

DataGrip did not mind the camel cased identity table names but vscode-database didn’t comprehend the table names when it was reading the database to build Intellisense for the tooling. It’s anticipating lowercase and throwing exceptions when it hits the camelcased names. Shay Rojansky, who maintains the npgsql PostgreSQL providers for EF & EF Core, explained to me that if the tool simply placed quotes around the names, it would be okay.

Update: The vscode-database maintainers have fixed the problem and you can now easily use the identity tables …just remember to put quotes around the camel-cased database objects when typing SQL. Although right now the installer hasn’t been updated with the new bits yet. I just fixed up the extension files manually for the time being.

If you prefer not to have to do that and do want the identity tables etc to be all lower case, then you may still find my hack to be useful.

For a temporary workaround,  I wanted to just change them to lower case so I could reap the benefits of vscode-database extension.

Since the IdentityDbContext class creates the table names in its OnModelCreating override, I needed to change those names to lower case after the fact.

It’s easy enough to do if you know the right path through the APIs.

Here’s my DbContext class that handles Identity in my ASP.NET Core app. I’ve added the foreach clause to lower case the name after the base class (IdentityDbContext) has already changed the names.

I’m grabbing the ToTable name that was specified by ApplicationDbContext and then re-applying the name as lower case.

using Microsoft.AspNetCore.Identity.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore;
using WebApplication.Models;

namespace WebApplication.Data {

  public class ApplicationDbContext : IdentityDbContext<ApplicationUser>
  {
    public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)
        : base(options) {
    }

    protected override void OnModelCreating(ModelBuilder builder) {
      base.OnModelCreating(builder);
      // Customize the ASP.NET Identity model and override the defaults if needed.
      // For example, you can rename the ASP.NET Identity table names and more.
      // Add your customizations after calling base.OnModelCreating(builder);
      //quick and dirty takes care of my entities not all scenarios
      foreach (var entity in builder.Model.GetEntityTypes()) {
        var currentTableName = builder.Entity(entity.Name).Metadata.Relational().TableName;
        builder.Entity(entity.Name).ToTable(currentTableName.ToLower());
      }
    }
  }
}

vscode-database has some cool features with Intellisense. It has some UX issues to work on (or maybe I need some training and I’ll raise an issue just in case) but it’s free and let’s you execute queries and commands on the database for those times you don’t need the bells and whistles of a full IDE .

EF Core Lets Us Finally Define NoTracking DbContexts

Way back in 2012, I added a feature request to EF6 to allow us to define a context that will never track entities that it retrieves from the database.

(Support Read-Only Context or DbSet)

This is instead of having to add AsNoTracking to all of your queries if you have a DbContext you are using for read-only data. Or more importantly, if your DbContext is being used for data that’s going to be disconnected and therefore never tracked post-query. That means a Web API or service or a controller. Tracking can be expensive if you are retrieving a lot of data and have no plans to update it. And having to remember to add AsNoTracking to every query is a PIA.

I just discovered that this is possible with EF Core and I think it was even in the EF Core 1.0 release!

There is a ChangeTracker property called QueryTrackingBehavior that takes a QueryTrackingBehavior enum whose options are NoTracking and TrackAll.

There are plenty of places to use it, but the one I’m excited about is to place it directly in the constructor of a DbContext to make that context default to never track any entities.

public class BookContext : DbContext
{
  public BooksReadOnlyContext() { 
    ChangeTracker.QueryTrackingBehavior = 
       QueryTrackingBehavior.NoTracking;
  }
 public DbSet<Book> Books {get;set;}
 etc ...

A quick test where I retrieved some books and the inspected the ChangeTracker.Entries returned 0, to show that this was doing what I’ve been dreaming of for over 5 years! Thanks EF team!

 Console.WriteLine(
   $"Tracked Entities: {context.ChangeTracker.Entries().Count()}");

Another point to be aware of is that just as you have always been able to use the DbSet’s AsNoTracking method to turn off tracking for a particular query, you can now use AsTracking to turn on tracking for a particular query.

Visual Studio Code Snippets to Make Coding EF Core a Little Simpler

I’ve been using the user snippet feature of Visual Studio Code to make it easier to get some of the code I commonly use for EF Core into my files. For example I have C# snippets for DbContext to create a constructor overload that takes in a DbContextOptions parameter, OnConfiguring or OnModeling . I have json snippets to add in the EFCore Commands dependency and the Tools section with EF Core tools.

github.com/julielerman/EFCoreSnippetsForVSCode

I finally created a github repository to share them. Since you’ll need to add the csharp snippets into your existing csharp snippets and the same with the json, i have put them into separate csharp.json and json.json files from which you can copy and past my snippets into your own.

Although the instructions are on the user defined snippets page I just linked to, the TL;DR is:

From menu, choose Code, Preferences then User Snippets

2016-10-30_11-56-58

That will open a list of snippet files. Choose C# for the C# snippets and Json for the Json snippets. Paste in my snippets!

2016-10-30_11-57-25

Completely New EF in the Enterprise Course on Pluralsight

My baby is here! A brand new Entity Framework in the Enterprise.

[See also:  New EF Core Course on Pluralsight!]

In 2012, I published a course on Pluralsight called Entity Framework in the Enterprise. Since then I have learned so much, most importantly, I’ve become very active with Domain-Driven Design, even publishing the DDD Fundamentals course that I co-created with Steve Smith. This has had a big impact on how I think about designing and architecting software and in turn,how I approach incorporating EF in to large, complex applications.

I’ve been wanting to re-do that old course to share my new views. I finally began in January of this year, but had a 3 month conference travel hiatus. So while it feels like a baby that I spent 9 months on, it was really only 6 months. Still quite a long time!

The course is now live! Entity Framework in the Enterprise

In the course, I use VS2015 and EF6. Why EF6? Because EF Core is too new. Most of the patterns I discuss and demonstrate are totally applicable to EF Core. There is one thing that is not yet in EF Core: Value Objects, but that is coming. Also the module on testing focuses on mocking and does not take EF Core’s new In Memory provider into account. Other than that, you can use what you learn here with EF Core as well.

I put a lot of thought into this course and I think this comment on the discussion forum for the course expresses it so well:

I just watched Julie Lerman’s prior Entity Framework in the Enterprise three weeks ago, before this new course was released, and boy am I glad she’s updated the course. I had thought the previous version was a bit dated (2016 vs 2012 & EF6 vs EF4) and a bit basic with what Julie refers to as Demo Ware. This updated course goes into more details about architecting projects, improved Moq testing with EF6, and a better explanation of DDD with Bounded Contexts using Schema to segregate areas. I already had a good understanding of EF, DDD, Repositories, UoW, and CQRS before watching and while I wouldn’t set up things 100% this way in my own applications they did jump start some refactorings and rethinking on how I maintain my solution, which is the purpose of these courses, to give fresh ideas as technology evolves, just as Entity Framework has. Thanks for updating the course and for those who have watched the previous version, definitely give this new one a watch.

image

New EF Core Course on Pluralsight

Pluralsight has recently published a Play by Play that I did with Geoffrey Grosenbach called EF Core 1.0 First Look.

[See also Completely New EF in the Enterprise Course!! Published Sept 23 2016]

This is a video of me showing EF Core to Geoffrey on my Mac. I spent 1.5 hours talking to him about EF Core, how it aligns with .NET Core’s which is cross platform, EF teams goals and some of the fun new things.

There are no slides. The video is 100% screen capture of my laptop while I’m creating a new ASP.NET Core Web API and adding EF Core into the APIs, connecting to a new PostgreSQL database, seeding the database, interacting with the data, testing with the new InMemory provider and even deploying to Docker.

The goal of this video is to get you, as it’s titled, a first look at EF Core. It also allowed me to get EF Core in front of you quickly while I now work on a “normal” course which is Getting Started with EF Core. This course will mostly be on Windows and Visual Studio but I will also flip over to Mac for a few demos at the end. The Getting Started course I’m working on will be a real step by step walkthrough, introductory course.

So in the meantime, take a look at this new Play by Play!

image

Upcoming EF Course on Pluralsight

9/14: I’ve been told “a few more business days”. Believe me I’m as eager as anyone can be for this to get released! 🙂

9/23: It’s here!!! 

https://app.pluralsight.com/library/courses/entity-framework-enterprise-update/table-of-contents

At the beginning of the year, I started dong a completely new version of my EF in the Enterprise course. That one is years old and was done with EF4 before DbContext and Code First even existed. Also before I started learning about Domain Driven-Design.

I had a lot of side-tracks in my schedule along the way including 3 months of conference travel and  a lot of hard thinking about how to explain and demonstrate some of these concepts. But I’ve finally finished the last module yesterday. I have some work to do in response to tech reviews of some of the module and I have to do the dreaded task of coming up with the questions for a few of the modules. But then it will be ready for Pluralsight to push through and get published. I don’t think it will be long now.

While I used EF6 for this course, most of the ideas also apply to EFCore as well.

In the meantime, I can tell you the titles of the 8 modules of this course which seems to have come out a little under 5 hours total:

  1. Architecting a Data Layer
  2. Understanding EF Encapsulation and the Great Repository Debates
  3. Implementing Encapsulation Patterns with EF6
  4. Managing Complex Domains and Data Models: Lessons from DDD Bounded Context
  5. Refactoring to Domain-Driven Design Bounded Contexts:A Walkthrough
  6. Handling the State of Disconnected Graphs
  7. Mapping DDD Domain Models with Entity Framework
  8. Testing Your Apps When Entity Framework is Involved

Watch this space for the new course: bit.ly/PS-Julie

Using JSON Data and EF Core to Seed a Database

I’m so used to use standard code (C# and EF APIs) to let EF help me seed a database. You know, instantiate an object, populate its fields, maybe add objects to a related list. Then add the whole kit n’ kaboodle to the DbContext and call SaveChanges.

I was showing some EF Core code to Geoffrey Grossenbach when we were talking about doing a Play by Play for Pluralsight on EF Core in advance of my buckling down to do a more serious course. Geoffrey looked at all the code for building up my objects to seed the database and said “wow that’s a lot of code. Can’t you use JSON or something?” (Note: I tired of trying to get this code formatted prettily in wordpress, but you get the point…right?)

private static List BuildWeatherEvents()
{
  var events = new List
  {
   WeatherEvent.Create(DateTime.Now,WeatherType.Sun,
    new List<string[]>{new []{"Julie","Oh so sunny!"}}),
   WeatherEvent.Create(DateTime.Now.AddDays(-2),WeatherType.Rain),
   WeatherEvent.Create(DateTime.Now.AddDays(-3),WeatherType.Sun,
    new List<string[]>{
      new []{"Julie","Oh lovely summer sun!
                      Too bad I'm on my computer"},
      new []{"Everyone in vermont", "Hooray let's go play!"},
      new []{"Sampson","I'd like to go for a swim, please!"},
    }),
   WeatherEvent.Create(DateTime.Now.AddDays(-4),WeatherType.Cloudy),
   WeatherEvent.Create(DateTime.Now.AddDays(-5),WeatherType.Rain),
   WeatherEvent.Create(DateTime.Now.AddDays(-6),WeatherType.Sun)
  };
 var lastEvent = 
   WeatherEvent.Create(DateTime.Now.AddDays(-1),  WeatherType.Snow,
          new List<string[]> {
             new[] { "Julie", "Snow? In July? 
                      Okay this is ridiculous even for VT!" } });
 lastEvent.Reactions.FirstOrDefault().Comments.Add
     (new Comment { Text = "Get over it, Julie!" });
 events.Add(lastEvent);
 return events;
}

Oh how much prettier it would be. Here’s how I’ve done it with EF Core but you can certainly use the same concept for doing the same with EF6.

My domain is WeatherEvent which is the domain in my EFCore demo at https://github.com/julielerman/EFCore-ASPNetCore-WebAPI-RTM, (which I have not yet updated to demonstrate using the JSON data).

Here’s the json which I store in a file called weatherdataseed.json.

[
 {
  "date": "2016-07-27T00:00:00",
  "time": "22:09:13.8216230",
  "type": 5,
  "reactions": [
   {
    "name": "Julie",
    "quote": "Oh so sunny!",
    "comments": []
   }
  ],
 "mostCommonWord": null
 },
 {
 "date": "2016-07-25T00:00:00",
 "time": "22:09:13.8237230",
 "type": 1,
 "reactions": [],
 "mostCommonWord": null
 },
 {
  "date": "2016-07-24T00:00:00",
  "time": "22:09:13.8238740",
  "type": 5,
  "reactions": [
   {
    "name": "Julie",
    "quote": "Oh lovely summer sun! Too bad I'm on my computer",
    "comments": []
   },
   {
    "name": "Everyone in vermont",
    "quote": "Hooray let's go play!",
    "comments": []
   },
   {
    "name": "Sampson",
    "quote": "I'd like to go for a swim, please!",
    "comments": []
   }
  ],
  "mostCommonWord": null
 },
 {
  "date": "2016-07-23T00:00:00",
  "time": "22:09:13.8239130",
  "type": 6,
  "reactions": [],
  "mostCommonWord": null
 },
 {
  "date": "2016-07-22T00:00:00",
  "time": "22:09:13.8239210",
  "type": 1,
  "reactions": [],
  "mostCommonWord": null
 },
 {
  "date": "2016-07-21T00:00:00",
  "time": "22:09:13.8239290",
  "type": 5,
  "reactions": [],
  "mostCommonWord": null
 },
 {
  "date": "2016-07-26T00:00:00",
  "time": "22:09:13.8239360",
  "type": 2,
  "reactions": [
   {
    "name": "Julie",
    "quote": "Snow? In July? Okay this is ridiculous even for VT!",
    "comments": [
     {
     "text": "Get over it, Julie!"
     }
    ]
   }
 ],
 "mostCommonWord": null
 }
]

So that’s not just data, but hierarchical data with 3 levels of relationship. Expressing it in json is a lot easier and prettier and readable than building all of that up in C#, creating the objects, etc.

Now of course it’s time for the magical JSON.NET which makes it possible to pull this data in to EF short and sweet.

This is the full code that I’m using to seed the database from the JSON using EF.

I’m calling it from startup.cs in an ASP.NET Core Web API inside the Configure method. Here I’m just reading the file with System.IO.File.ReadAllText and then passing that text into my Seedit method. Also note the ConfigureServices where I’m setting up the DbContext along with the connection string it requires.

Note: There’s been some churn in the code in this post as  Shawn Wildermuth and I went through some learning on twitter with Dave Fowler, one of the core leads on the aspnet team. I  changed my original code to streamline it as per a great suggestion from Shawn, but Dave pointed out some scoping issues with that. So now the sample is back to my original version, where the seedit method is responsible for creating a new ServiceScope and instantiating the context. 

 public void ConfigureServices(IServiceCollection services)
 { 
   services.AddDbContext<WeatherContext>(
     options=> options.UseNpgsql(
       Configuration["Data:PostgreConnection:ConnectionString"]));
   services.AddMvc();
 }

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
  loggerFactory.AddConsole(Configuration.GetSection("Logging"));
  loggerFactory.AddDebug();

  app.UseMvc();
  var dataText=System.IO.File.ReadAllText(@"weatherdataseed.json");
  Seeder.Seedit(dataText,app.ApplicationServices);
}

Here’s the Seedit method, which uses JSON.NET to deserialize the json into a list of  WeatherEvent objects.  The contract serializer is to overcome private setters in my WeatherEvent class. I got this from Daniel Wertheim’s github repo. Then I use the ASP.NET Core ServiceProvider to get service I set up in startup which will instantiate a WeatherContext along with the connection string specified in startup.

using System;
using System.Linq;
using System.Collections.Generic;
using Microsoft.Extensions.DependencyInjection;
using EFCoreWebAPI;
using Newtonsoft.Json;
using JsonNet.PrivateSettersContractResolvers;

public static class Seeder {
  public static void Seedit(string jsonData,
                            IServiceProvider serviceProvider) {
  JsonSerializerSettings settings = new JsonSerializerSettings {
    ContractResolver = new PrivateSetterContractResolver()
  };
  List<WeatherEvent> events =
   JsonConvert.DeserializeObject<List<WeatherEvent>>(
     jsonData, settings);
  using (
   var serviceScope = serviceProvider
     .GetRequiredService<IServiceScopeFactory>().CreateScope())
   {
     var context = serviceScope
                   .ServiceProvider.GetService<WeatherContext>();
     if (!context.WeatherEvents.Any()) {
       context.AddRange(events);
       context.SaveChanges();
     }
   }
 }
}

After instantiating the context, I have it check to see if I have any weather events in the database yet so I don’t re-seed. (This is logic I want for seeding during my demo so you may have different rules for the seeding.) Now I call context.AddRange passing in the list of WeatherEvent objects. I could use the DbSet directly or the context to call AddRange. And finally, SaveChanges.

So the key here, whether you are using EFCore or EF6 (so the use of the service is specific to the fact that my app is an ASPNET Core api), is really just reading the json file, deserializing the data and adding it to the context. This is oh, so much simpler than creating all of that data imperatively.

It may not always be the answer depending on the shape of your data but it was perfect for this particular model and the seed data that I needed.

 

What’s Coming Up in EFCore’s Near Future

EF Core 1.0.0 was released June 27 along with ASP.NET Core.

Even before that date, more goodies were added and fixes were made that were not able to get into that package. For example, we’ve got DbSet.Find back now in the nightly builds and a fix for a big performance problem [that affected a narrow set of use cases which is probably why it went undetected for so long] which related to the asynchronous eager loading (Include). It was fixed within a few days of the issue being created.

There is a patch (1.0.1) coming hopefully in early August. (From dotnet blog linked below: “There is no scheduled date for this patch update but early August is likely.” ) But that is only a patch to fix important bugs. (Maybe the async include performance fix will be in there but Rowan Miller explicitly said “Find” is in 1.1, not 1.0.1.)

2016-07-25_13-51-30

1.1 is feature release and is planned for possibly Q4 2016. This is the one where we go from project.json back to csproj/msbuild.  From the mid-July dotnet blog post titled .NET Core Roadmap , below is the list of the important features that will come to EF Core at that time.


If you don’t see what you are looking for I have two suggestions:

  1. Look at the EF Core roadmap on its Github repository. You can find it at bit.ly/efcoreroadmap.
  2. Search the github repository. For example, here’s a search for those of you looking for TPT conversations

Entity Framework Core Minor Update (1.1?)
Q4 2016 / Q1 2017

  • Azure
    • Transient fault handling (resiliency)
  • Mapping
    • Custom type conversions
    • Complex types (value objects)
    • Entity entry APIs
  • Update pipeline
    • CUD stored procedures
    • Better batching (TVPs)
    • Ambient transactions
  • Query
    • Stability, performance.
  • Migrations
    • Seed data
    • Stability
  • Reverse engineer
    • Pluralization
    • VS item template (UX)