Tag Archives: EF

EF Core CLI Commands with VS2017 RC3

Visual Studio 2017 RC3 was released yesterday but unfortunately an install issue has take it back off the shelf for a brief period. Watch this space for the return of RC3!

But I did manage to get it installed and wanted to show you that the EF Core CLI commands are now working. If you’ve been playing with VS2017 RC and EF Core you may have run into the problem that the EF Core tooling package was not in sync yet with the MSBuild tooling for .NET Core. That’s fixed now and not only does it work but there’s a change that I’m really happy to see.

As always, I have my dbcontext in its own project. Here are the csproj contents for that project:

image

 

Notice that the DotNetCliToolReference is pointing to Microsoft.EntityFrameworkCore.Tools.DotNet . The dotnet and PowerShell commands are exposed in separate packages. With “.DotNet” is the package that has the CLI commands. Without “.DotNet” is the package that contains the PowerShell commands.

More importantly, the package version went from “1.1.0-preview4” to “1.0.0-msbuild3-final”. I can’t explain why we went from 1.1.0 down to 1.0.0 but this is the newer and correct package.

With that in place,  I then open up a command prompt. I can use a regular one but I’m using a PowerShell command for a single benefit…that I can shorten the prompt. Here’s the command I did to trim most but not all of the path:

Quora: How do I get just the current folder name in my Windows Powershell prompt function?

Remember that I’m pointed to the path of a class library. DotNet EF requires you to point to a path containing an executable in order to run the commands. However with the latest bits, you can get HELP without having to point to the executable. Thank you Brice Lambson. It was a little meta to have to figure that out because rather than just getting help from the command, you had to googlebing for help on how to get help. So here are a simple dotnet ef command to get top level dotnet ef help, followed by dotnet ef dbcontext to get help on the dbcontext sub-commands.

image

To run commands that depend on the APIs, you still have to point to a startup-project if you are running the commands from a class library. Here I’ve run the command to list the migrations in my project. I’ve only got one, sqlite-init.

image

Some Insights into Features (Besides EDMX) Being Dropped in the Move From EF6 to EF Core

I had written these details for my Pluralsight EF Core course (hopefully published at end of Jan 2017) but decided not to spend the time on this explanation in the course. Instead, I’ll put it here and the course will have a link to this blog post! Clever, huh?

You’ve probably heard a lot about EF Core not bringing forward  the designer based EDMX from EF Core. This is a feature cut that I’ve heard the most feedback about. But there are other EF features that are also getting cut. These are not as worrisome to developers — based on my own experience and paying attention to response in social media — but it’s important to be aware of the biggest of these cut features.

ObjectContext API

The first is the ObjectContext API. This was the original mechanism for EF’s change tracking and database interaction. Since EF4.1 was released with the DbContext in early 2011, Microsoft has recommended that all new projects use DbContext. The DbContext sits on top of the ObjectContext and does the more cumbersome work of interacting with the ObjectContext on your behalf. But the ObjectContext has remained a public API for backwards compatibility with EF4 and EF3.5 projects. Also, we could access the ObjectContext to do low level tasks as needed.

There will not be an ObjectContext API in EF Core. Rather than relying on the ObjectContext for metadata work, change tracking and database interaction, this low-level activity is being  restructured and we’ll get at it directly from the DbContext. If you have old software that is still using the ObjectContext and you haven’t updated it by now, hopefully, you won’t want to update it to EFCore anyway. I wrote a 2 part article for MSDN magazine in 2014 that included guidance for moving ObjectContext code to DbContext if you think you may want to explore that.

Data Points : Tips for Updating and Refactoring Your Entity Framework Code Part 1

Data Points : Tips for Updating and Refactoring Your Entity Framework Code Part 2

Entity SQL

Entity SQL was the original string-based querying SQL like language written for EF. By the time EF was first released, it had already embraced the also-new LINQ. ESQL is only usable with the ObjectContext API. I think I  used to be one of the few people in the world who really knew how to use ESQL because I wrote about it extensively in my first EF book, giving it equal visibility as LINQ to Entities. In the 2nd edition, I had split the ESQL details out into their own chapter because by then it was clear that it was barely being used. I haven’t had any reason to use ESQL in many years. I’ve not heard of anyone using it either. So it is going to fade away along with the ObjectContext API and won’t be part of EFCore.

Edge-Case Mappings & the Original Metadata APIs

Entity Framework has allowed a lot of variations on mappings between your classes/properties and your database tables/fields. It has even let you combine many of these crazy mappings in one model. The EF team blog post highlights and example: “an inheritance hierarchy that combined TPH, TPT, and TPC mappings as well as Entity Splitting all in the same hierarchy.” This was possible because of the Metadata Workspace API. But building in this flexibility also meant that using that API was very complex. Internal query compilation was difficult to design. And for developers, discovering information about a model’s metadata has been very cumbersome.

So, EFCore has a simpler metadata model which means some of the truly edge case mappings won’t be achievable. This doesn’t mean things like inheritance will go away (although currently, EF Core only supports TPH), just the funky, rare mapping combinations.

MEST (Multiple Entities for a Single Type)

One single mapping technique that will go away is MEST. In all of my years of working with EF, I’ve never come across anyone who was taking advantage of it. It was only supported with EDMX and ObjectContext and the team decided not to bring it forward to the code-based model and DbContext for EFCore.

Automatic Migrations

Migrations are a critical technique for evolving a database schema from a code-based model. We’ve had two ways to use EF migrations – the default way which is to explicitly add migrations through the package manager console and then to apply those migrations using a variety of techniques. Another option has been to use automatic migrations that are worked out and executed on the fly at run time. Supporting automatic migrations caused a number of major headaches for migrations support overall. It forced migrations to store model snapshots directly in the database. This caused problems for developers using regular migrations – especially with source control. I’ve been in loops where I can’t add a migration because it thinks I need to execute one, but when I try to execute a migration it tells me I have to add one. I’m  not the only one who has gotten all tangled up in some circular problems when trying to manage migrations. These problems will go away because EFCore will not attempt to automate migrations at all. You can read more about this in Brice Lambson’s blog post about EFCore Migrations at design time. Brice is an engineer on the EF team and has a lot of other interesting blog posts worth checking out.

These are the most notable EF features that the team is not planning to implement at all in EFCore. Personally, I have not been using any of them ever or in a long time and I have guided my clients away from them as well. So if you are on that same path, you are well-positioned to use EF Core without having to worry about them.

Updated My EFCore / WebAPI / PostreSQL / XUnit Repo to 1.1

Today was dedicated to updating my long running repository sample that I started when EF Core was EF 7  to the newest version of EF Core: 1.1. Here is the updated repo: https://github.com/julielerman/EFCore-ASPNetCore-WebAPI-RTM.

Phase one of this update continues to use project.json.

In addition to updating the version #s of  the Nuget package references, I also made some changes to the code to reflect a few new features.

Pay attention to the tooling packages. In the tools section, the package name has changed – note DotNet at the end –  and the version is currently 1.0.0-preview3 even though IIS version is preview2.

 "tools": {
   "Microsoft.AspNetCore.Server.IISIntegration.Tools": 
       "1.0.0-preview2-final",
   "Microsoft.EntityFrameworkCore.Tools.DotNet":
       "1.0.0-preview3-final"
 },

Also in the dependencies, the EFCore Design package is 1.1.0 like the rest of EFCore. That’s part of the EF APIs, not tooling.

Code changes ….

You’ll discover the DbSet.Find method and change tracker Load method in use in the repository class. These were both added in to EF Core 1.1.

I modified the WeatherEvent class to fully encapsulate its Reactions collection using the support for mapping to IEnumerable. That resulted in some changes to constructors and the addition of an AddReaction method and a local variable.

Unrelated to EF Core, I also modified the SeedData.cs class. It reads a hard coded seeddata.json file to read in seed data. That data used old dates. I wanted the data to show current dates to help me tell that I really and truly pushed new data into the database. Since the Date property of WeatherEvent is private, they way I went about this was to read the raw JSON and update the date value that way then save the raw JSON back to the original file. Then I deserialize the JSON with a current range of dates into a set of WeatherEvents. This also means that I added Delete/Create database back in so the database gets thrown away and recreated/reseeded every time you start up the application.

The tests are also update to use the latest packages. In addition to changing the versions, I had to add a reference to an older package (InternalServices) as its dependency has not yet been updated in xunit.

Here’s the full project.json for the test project since I had to do a bunch of googling to figure it out.

{
 "version": "3.0.0-*",
 "description": "Tests for simple app using aspnetcore, efcore and   
                  postgresql. developed and run on OSX.",
 "authors": [ "Julie Lerman" ],
 "testRunner": "xunit",
 "dependencies": {
   "Microsoft.EntityFrameworkCore.InMemory": "1.1.0",
   "src": "3.0.0",
   "xunit": "2.2.0-beta4-build3444",
   "dotnet-test-xunit": "2.2.0-preview2-build1029",
   "Microsoft.DotNet.InternalAbstractions":"1.0.0"},
 "frameworks": {
 "netcoreapp1.0": {
   "dependencies": {
     "Microsoft.NETCore.App": {
     "type": "platform",
     "version": "1.1.0"
     }
   },
   "imports": [
     "dnxcore50",
     "portable-net45+win8"
     ]
   }
  }
}

I hope you find this repository useful to see EF Core 1.1 in action.

Oh and as per a tweet by Brad Wilson, I added SDK to my global.json file!

Now I have to go learn about why this is important. Clearly it is!

 

EF6 or EF Core? How Do I Choose?

In late October, I spoke at DevIntersection. One of my sessions was called EF6 or EF Core? How Do I Choose? As I’ve been an authority on EF since it’s earliest days, people have been asking me this question frequently, which inspired the talk.

The session was not recorded but I’m sharing my slides on SlideShare. I go into detail on the topic in my upcoming Pluralsight course, “EF Core: Getting Started” which I am currently building. Watch my Pluralsight author page (as well as twitter.com/julielerman and this blog!) for its release.

In the meantime, here is a link to the slides from the DevIntersection talk.ef6-or-ef-core

Completely New EF in the Enterprise Course on Pluralsight

My baby is here! A brand new Entity Framework in the Enterprise.

[See also:  New EF Core Course on Pluralsight!]

In 2012, I published a course on Pluralsight called Entity Framework in the Enterprise. Since then I have learned so much, most importantly, I’ve become very active with Domain-Driven Design, even publishing the DDD Fundamentals course that I co-created with Steve Smith. This has had a big impact on how I think about designing and architecting software and in turn,how I approach incorporating EF in to large, complex applications.

I’ve been wanting to re-do that old course to share my new views. I finally began in January of this year, but had a 3 month conference travel hiatus. So while it feels like a baby that I spent 9 months on, it was really only 6 months. Still quite a long time!

The course is now live! Entity Framework in the Enterprise

In the course, I use VS2015 and EF6. Why EF6? Because EF Core is too new. Most of the patterns I discuss and demonstrate are totally applicable to EF Core. There is one thing that is not yet in EF Core: Value Objects, but that is coming. Also the module on testing focuses on mocking and does not take EF Core’s new In Memory provider into account. Other than that, you can use what you learn here with EF Core as well.

I put a lot of thought into this course and I think this comment on the discussion forum for the course expresses it so well:

I just watched Julie Lerman’s prior Entity Framework in the Enterprise three weeks ago, before this new course was released, and boy am I glad she’s updated the course. I had thought the previous version was a bit dated (2016 vs 2012 & EF6 vs EF4) and a bit basic with what Julie refers to as Demo Ware. This updated course goes into more details about architecting projects, improved Moq testing with EF6, and a better explanation of DDD with Bounded Contexts using Schema to segregate areas. I already had a good understanding of EF, DDD, Repositories, UoW, and CQRS before watching and while I wouldn’t set up things 100% this way in my own applications they did jump start some refactorings and rethinking on how I maintain my solution, which is the purpose of these courses, to give fresh ideas as technology evolves, just as Entity Framework has. Thanks for updating the course and for those who have watched the previous version, definitely give this new one a watch.

image

Upcoming EF Course on Pluralsight

9/14: I’ve been told “a few more business days”. Believe me I’m as eager as anyone can be for this to get released! 🙂

9/23: It’s here!!! 

https://app.pluralsight.com/library/courses/entity-framework-enterprise-update/table-of-contents

At the beginning of the year, I started dong a completely new version of my EF in the Enterprise course. That one is years old and was done with EF4 before DbContext and Code First even existed. Also before I started learning about Domain Driven-Design.

I had a lot of side-tracks in my schedule along the way including 3 months of conference travel and  a lot of hard thinking about how to explain and demonstrate some of these concepts. But I’ve finally finished the last module yesterday. I have some work to do in response to tech reviews of some of the module and I have to do the dreaded task of coming up with the questions for a few of the modules. But then it will be ready for Pluralsight to push through and get published. I don’t think it will be long now.

While I used EF6 for this course, most of the ideas also apply to EFCore as well.

In the meantime, I can tell you the titles of the 8 modules of this course which seems to have come out a little under 5 hours total:

  1. Architecting a Data Layer
  2. Understanding EF Encapsulation and the Great Repository Debates
  3. Implementing Encapsulation Patterns with EF6
  4. Managing Complex Domains and Data Models: Lessons from DDD Bounded Context
  5. Refactoring to Domain-Driven Design Bounded Contexts:A Walkthrough
  6. Handling the State of Disconnected Graphs
  7. Mapping DDD Domain Models with Entity Framework
  8. Testing Your Apps When Entity Framework is Involved

Watch this space for the new course: bit.ly/PS-Julie

Using JSON Data and EF Core to Seed a Database

I’m so used to use standard code (C# and EF APIs) to let EF help me seed a database. You know, instantiate an object, populate its fields, maybe add objects to a related list. Then add the whole kit n’ kaboodle to the DbContext and call SaveChanges.

I was showing some EF Core code to Geoffrey Grossenbach when we were talking about doing a Play by Play for Pluralsight on EF Core in advance of my buckling down to do a more serious course. Geoffrey looked at all the code for building up my objects to seed the database and said “wow that’s a lot of code. Can’t you use JSON or something?” (Note: I tired of trying to get this code formatted prettily in wordpress, but you get the point…right?)

private static List BuildWeatherEvents()
{
  var events = new List
  {
   WeatherEvent.Create(DateTime.Now,WeatherType.Sun,
    new List<string[]>{new []{"Julie","Oh so sunny!"}}),
   WeatherEvent.Create(DateTime.Now.AddDays(-2),WeatherType.Rain),
   WeatherEvent.Create(DateTime.Now.AddDays(-3),WeatherType.Sun,
    new List<string[]>{
      new []{"Julie","Oh lovely summer sun!
                      Too bad I'm on my computer"},
      new []{"Everyone in vermont", "Hooray let's go play!"},
      new []{"Sampson","I'd like to go for a swim, please!"},
    }),
   WeatherEvent.Create(DateTime.Now.AddDays(-4),WeatherType.Cloudy),
   WeatherEvent.Create(DateTime.Now.AddDays(-5),WeatherType.Rain),
   WeatherEvent.Create(DateTime.Now.AddDays(-6),WeatherType.Sun)
  };
 var lastEvent = 
   WeatherEvent.Create(DateTime.Now.AddDays(-1),  WeatherType.Snow,
          new List<string[]> {
             new[] { "Julie", "Snow? In July? 
                      Okay this is ridiculous even for VT!" } });
 lastEvent.Reactions.FirstOrDefault().Comments.Add
     (new Comment { Text = "Get over it, Julie!" });
 events.Add(lastEvent);
 return events;
}

Oh how much prettier it would be. Here’s how I’ve done it with EF Core but you can certainly use the same concept for doing the same with EF6.

My domain is WeatherEvent which is the domain in my EFCore demo at https://github.com/julielerman/EFCore-ASPNetCore-WebAPI-RTM, (which I have not yet updated to demonstrate using the JSON data).

Here’s the json which I store in a file called weatherdataseed.json.

[
 {
  "date": "2016-07-27T00:00:00",
  "time": "22:09:13.8216230",
  "type": 5,
  "reactions": [
   {
    "name": "Julie",
    "quote": "Oh so sunny!",
    "comments": []
   }
  ],
 "mostCommonWord": null
 },
 {
 "date": "2016-07-25T00:00:00",
 "time": "22:09:13.8237230",
 "type": 1,
 "reactions": [],
 "mostCommonWord": null
 },
 {
  "date": "2016-07-24T00:00:00",
  "time": "22:09:13.8238740",
  "type": 5,
  "reactions": [
   {
    "name": "Julie",
    "quote": "Oh lovely summer sun! Too bad I'm on my computer",
    "comments": []
   },
   {
    "name": "Everyone in vermont",
    "quote": "Hooray let's go play!",
    "comments": []
   },
   {
    "name": "Sampson",
    "quote": "I'd like to go for a swim, please!",
    "comments": []
   }
  ],
  "mostCommonWord": null
 },
 {
  "date": "2016-07-23T00:00:00",
  "time": "22:09:13.8239130",
  "type": 6,
  "reactions": [],
  "mostCommonWord": null
 },
 {
  "date": "2016-07-22T00:00:00",
  "time": "22:09:13.8239210",
  "type": 1,
  "reactions": [],
  "mostCommonWord": null
 },
 {
  "date": "2016-07-21T00:00:00",
  "time": "22:09:13.8239290",
  "type": 5,
  "reactions": [],
  "mostCommonWord": null
 },
 {
  "date": "2016-07-26T00:00:00",
  "time": "22:09:13.8239360",
  "type": 2,
  "reactions": [
   {
    "name": "Julie",
    "quote": "Snow? In July? Okay this is ridiculous even for VT!",
    "comments": [
     {
     "text": "Get over it, Julie!"
     }
    ]
   }
 ],
 "mostCommonWord": null
 }
]

So that’s not just data, but hierarchical data with 3 levels of relationship. Expressing it in json is a lot easier and prettier and readable than building all of that up in C#, creating the objects, etc.

Now of course it’s time for the magical JSON.NET which makes it possible to pull this data in to EF short and sweet.

This is the full code that I’m using to seed the database from the JSON using EF.

I’m calling it from startup.cs in an ASP.NET Core Web API inside the Configure method. Here I’m just reading the file with System.IO.File.ReadAllText and then passing that text into my Seedit method. Also note the ConfigureServices where I’m setting up the DbContext along with the connection string it requires.

Note: There’s been some churn in the code in this post as  Shawn Wildermuth and I went through some learning on twitter with Dave Fowler, one of the core leads on the aspnet team. I  changed my original code to streamline it as per a great suggestion from Shawn, but Dave pointed out some scoping issues with that. So now the sample is back to my original version, where the seedit method is responsible for creating a new ServiceScope and instantiating the context. 

 public void ConfigureServices(IServiceCollection services)
 { 
   services.AddDbContext<WeatherContext>(
     options=> options.UseNpgsql(
       Configuration["Data:PostgreConnection:ConnectionString"]));
   services.AddMvc();
 }

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
  loggerFactory.AddConsole(Configuration.GetSection("Logging"));
  loggerFactory.AddDebug();

  app.UseMvc();
  var dataText=System.IO.File.ReadAllText(@"weatherdataseed.json");
  Seeder.Seedit(dataText,app.ApplicationServices);
}

Here’s the Seedit method, which uses JSON.NET to deserialize the json into a list of  WeatherEvent objects.  The contract serializer is to overcome private setters in my WeatherEvent class. I got this from Daniel Wertheim’s github repo. Then I use the ASP.NET Core ServiceProvider to get service I set up in startup which will instantiate a WeatherContext along with the connection string specified in startup.

using System;
using System.Linq;
using System.Collections.Generic;
using Microsoft.Extensions.DependencyInjection;
using EFCoreWebAPI;
using Newtonsoft.Json;
using JsonNet.PrivateSettersContractResolvers;

public static class Seeder {
  public static void Seedit(string jsonData,
                            IServiceProvider serviceProvider) {
  JsonSerializerSettings settings = new JsonSerializerSettings {
    ContractResolver = new PrivateSetterContractResolver()
  };
  List<WeatherEvent> events =
   JsonConvert.DeserializeObject<List<WeatherEvent>>(
     jsonData, settings);
  using (
   var serviceScope = serviceProvider
     .GetRequiredService<IServiceScopeFactory>().CreateScope())
   {
     var context = serviceScope
                   .ServiceProvider.GetService<WeatherContext>();
     if (!context.WeatherEvents.Any()) {
       context.AddRange(events);
       context.SaveChanges();
     }
   }
 }
}

After instantiating the context, I have it check to see if I have any weather events in the database yet so I don’t re-seed. (This is logic I want for seeding during my demo so you may have different rules for the seeding.) Now I call context.AddRange passing in the list of WeatherEvent objects. I could use the DbSet directly or the context to call AddRange. And finally, SaveChanges.

So the key here, whether you are using EFCore or EF6 (so the use of the service is specific to the fact that my app is an ASPNET Core api), is really just reading the json file, deserializing the data and adding it to the context. This is oh, so much simpler than creating all of that data imperatively.

It may not always be the answer depending on the shape of your data but it was perfect for this particular model and the seed data that I needed.

 

What’s Coming Up in EFCore’s Near Future

EF Core 1.0.0 was released June 27 along with ASP.NET Core.

Even before that date, more goodies were added and fixes were made that were not able to get into that package. For example, we’ve got DbSet.Find back now in the nightly builds and a fix for a big performance problem [that affected a narrow set of use cases which is probably why it went undetected for so long] which related to the asynchronous eager loading (Include). It was fixed within a few days of the issue being created.

There is a patch (1.0.1) coming hopefully in early August. (From dotnet blog linked below: “There is no scheduled date for this patch update but early August is likely.” ) But that is only a patch to fix important bugs. (Maybe the async include performance fix will be in there but Rowan Miller explicitly said “Find” is in 1.1, not 1.0.1.)

2016-07-25_13-51-30

1.1 is feature release and is planned for possibly Q4 2016. This is the one where we go from project.json back to csproj/msbuild.  From the mid-July dotnet blog post titled .NET Core Roadmap , below is the list of the important features that will come to EF Core at that time.


If you don’t see what you are looking for I have two suggestions:

  1. Look at the EF Core roadmap on its Github repository. You can find it at bit.ly/efcoreroadmap.
  2. Search the github repository. For example, here’s a search for those of you looking for TPT conversations

Entity Framework Core Minor Update (1.1?)
Q4 2016 / Q1 2017

  • Azure
    • Transient fault handling (resiliency)
  • Mapping
    • Custom type conversions
    • Complex types (value objects)
    • Entity entry APIs
  • Update pipeline
    • CUD stored procedures
    • Better batching (TVPs)
    • Ambient transactions
  • Query
    • Stability, performance.
  • Migrations
    • Seed data
    • Stability
  • Reverse engineer
    • Pluralization
    • VS item template (UX)