Category Archives: Data Access

EF Code First Migrations, Update-Database outside of Visual Studio

In a recent blog post comment, someone asked “can you please tell them [EF Dev Team] some developers would like to use the Power Shell command script and not PM console to update database.”

If you look inside the package folder for Entity Framework 4.3 (or 4.3.1 or whatever is the current version in the future) there’s a tools directory and inside of there a migrate.exe command. It is the same as the update-database command with all of the same parameters.

migratecommandline

Entity Framework Moves to the Big House

Modified the original title of this post because it was derived from a misquote. Scott Hanselman clarifies with: The quote (what I said was) "EF is joining our team under ScottGu" and "EF is now the M in MVC". Scott Guthrie chimes in with “Yes – now part of my team (which includes ASPNET, WCF, WF, EF, Service Bus, BizTalk, Cache)”

_________________________________

Arthur Vickers, from the EF team, tweeted:

Also, very happy to be moving to @scottgu’s org with the rest of the Entity Framework team. Great for us, great for EF, great for Microsoft.

Just a reminder of Scott’s current role: “responsible for delivering the development platform for Windows Azure, as well as the .NET Framework and Visual Studio technologies used in building Web and server applications.”

A little later there was this on twitter from Noah Coad who is at Dallas Days of .NET (follow #dodn12 on twitter)

@shanselman at #dodn12 "just announced, we have an M in MVC, Entity Framework is now part of ASP.NET"

And then this interesting question from Daniel Bradley:

if Entity framework is to be part of ASP.NET, does this mean open-sourcing of the tools & framework?

Stay tuned….

(and since someone asked, EF has always been part of ADO.NET)

New EF Code First Migrations Video on Pluralsight

My newest course is live on Pluralsight.com. This one is about using the new data migrations feature of Entity Framework Code First. These were first introduced with EF 4.3. Migrations lets you update your database schema when your model changes rather than having code first completely drop and recreate the database. This lets you retain data and any other existing database objects that aren’t described in the model (e.g., triggers, functions, etc.)

Entity Framework Code First Migrations
(total 1 hr 09 m)

  • Module 1: Introduction and Automatic Migrations
  • Module 2: Code-Based Migrations

Here’s the entire list of EF courses I’ve done for Pluralsight (so far Smile). I’ve sorted them in the order you might want to watch them in.

Entity Framework (8 courses)

Entity Framework 4.0 By Example
Beginner
[02:18:50]
27 Aug 2010

Entity Framework and Data Models
Intermediate
[01:31:38]
5 Nov 2010

Designer Supported EDM Customization
Intermediate
[02:05:13]
6 Jan 2011

Querying the Entity Framework
Intermediate
[01:22:37]
28 Feb 2011

Entity Framework 4.1 – Code First
Intermediate
[01:58:17]
18 Jun 2011

Entity Framework 4.1 – DbContext Data Access
Intermediate
[01:21:19]
27 Sep 2011

Data Layer Validation with Entity Framework 4.1+
Intermediate
[01:53:49]
8 Feb 2012

Entity Framework Code First Migrations
Intermediate
[01:09]
6 Mar 2012

Take care with EF 4.3 AddOrUpdate Method

EF 4.3 added a new extension method for DbSet called AddOrUpdate.

It is meant for use with seeding data during migrations.

It looks like a nice method to add into your apps but that’s not it’s purpose.

The job of AddOrUpdate is to ensure that you don’t create duplicates when you seed data during development.

First, it will execute a query in your database looking for a record where whatever you supplied as a key (first parameter) matches the mapped column value (or values)  supplied in the AddOrUpdate. So this is a little loosey-goosey for matching but perfectly fine for seeding design time data.

More importantly, if a match is found then the update will update all and null out any that weren’t in your AddOrUpdate.

For example, if your type is:

Person

  • Id
  • Name
  • UserName
  • CreateDate
  • Bio
  • Email

And there is a row in the database with

5, GiantPuppy, Sampson, 1/1/2011, I drool, gp@notarealdomain.com

And your AddOrUpdate has

context.Aliases.AddOrUpdate(
           a => a.Name,
           new Alias { Name = "GiantPuppy", UserName = "Sampson", CreateDate = DateTime.Now},
           new Alias { Name = "JulieLerman", UserName = "Julie", CreateDate = DateTime.Now}
         );

The method will look for a row where whatever column maps to the Name property has the value GiantPuppy. It will then update all of the mapped properties.

  • Name: GiantPuppy
  • UserName: Sampson
  • CreateDate: whatever the time is
  • Bio: null  (because it wasn’t provided)
  • Email: null (because it wasn’t provided)

This is about the same as doing an update in a disconnected app where you attach an object, mark it as modified and call SaveChanges. I see people do this in MVC apps all the time. If there are properties that were left null in the object because you didn’t care about them in this particular case, the database values will get overwritten with null. Except this is worse because it’s not leaning on your entity key (e.g. Id mapped to primary key).

So be careful out there. Me? I’m planning to use AddOrUpdate only for seeding migrations!

Update: I thought it would be useful to answer Livingston’s question here in the main post.

The update will update all of the properties it sees as changed.

It queries for the match (name==GiantPuppy), compares the values in the AddOrUpdate command to the results and updates anything that’s different.

So the results of the query would return the Id field for Giantpuppy. The migration will see that GiantPuppy’s seed Bioi(null) is different than the database Bio (“I drool”), that the email (null) is different than the database email and that the createdate value is also different. 

Here is the update command it sends to the database:

exec sp_executesql N’update [dbo].[Aliases]
set  [Email] = null, [Bio] = null, [CreateDate] = @0
where ([Id] = @1)
from [dbo].[Aliases]
where @@ROWCOUNT > 0 and [Id] = @1′,
N’@0 datetime2(7),@1 int’,@0=’2012-02-29 20:57:23.2779868′,@1=3

It’s updating everything that’s different…. email, bio and CreateDate.

Using EF 4.3 Code First Migrations with an Existing Database

In working on my upcoming EF 4.3 Migrations video for Pluralsight, I wanted to work out how to use this with an existing database where I plan to add new types and therefore want migrations to not just use this database, but migrate it as well. Problem solved*, but then I tried to use it in a production application and found an easy-to-fix problem. So…I thought I’d share the process in a blog post while it’s on my mind.

The Purpose of the Migration-History Table

When you let EF 4.3 code first create a database for you, it inserts a Migration-History table into the new database. It’s hidden in System Tables. This is equivalent (in a way) to the EdmMetadata table you would have gotten with EF 4.1 & 4.2.

The table is used by two processes (maybe more that I haven’t encountered yet).

When you run the app using automatic migrations, if it’s doing any type of data initialization, it will check that Migration-History table to see if the database needs to be updated.

If you are using code-based migrations, Update-Database will also look at the Migration-History table.

If there’s no migration-history table (which there won’t be in an existing database), the model won’t be verified against the database with automatic migrations.

So you need to get that table into your existing database.

That table is created using the most recent migration file in your data layer.

Getting a Migration-History table into an existing database

If you’re starting with a new app, you won’t have that either.

So

Step 1) In Package Manager Console, execute “add-migration initial”.

This will force code first to create a migration class (which I chose to name “initial”) based on the code first model. It will have migration code to create all of the necessary database tables (based on what it’s found in the model) with their columns, keys and constraints.

But your database already exists! If you were to try to execute that migration and you have entities that map to those existing tables, EF will try to create the tables and tell you that the tables already exist.

Step 2) remove all code that duplicates existing tables from inside the Up and Down override methods in the initial migration. But, leave the methods there.

This is a little tricky. I find that it’s safer to do this when there’s nothing in my model yet that would required a database modification. That way it’s all clean. I’ve done this when I had one new class and then I had to start over again to get the true “initial database” and then start making any mods I want to my model.

Now, code first has a migration that contains no database modification code. That represents how things are at the beginning of EF 4.3 migrations involvement in your app.

Step 3) run Update-Database

This will do two things. It will execute the Up method in the initial migration, which has no effect on the database. And it will create the Migration-History table in your database based on the current state of the model. That way, if you are using automatic migrations, next time Code First does db initialization, it will compare the current model to the one stored in the migration-history to determine if the database needs to be migrated/updated.

Now you can evolve your app and let code first fix up your development database as needed.

The problem I encountered? A SQL Server 2000 Database

I was doing this with an existing client database (not production, but a copy for development … just sayin’  Winking smile)

When running Update-Database I got the following error:

Conversion failed when converting date and/or time from character string.

Remember I’m not really updating my database. This was when code first was trying to create the Migration-History table.

It is an old database that has been recently updated to SS2008. Even though I’m using SQL Server 2008R2, that database was still set as a SQL Server 2000 database and was not happy with the format of the date field in the INSERT command:

INSERT INTO [__MigrationHistory] 
([MigrationId], [CreatedOn], [Model], [ProductVersion])
VALUES ('201202161546124_initial', '2012-02-16T15:55:56.252Z',
[big-ass hash you don’t need to see], '4.3.0')

Luckily, I was able to just update my database version to 2008  and the insert command succeeded with no problem.

*still finding some more interesting tasks I have to do. For example, as I ensure that my model mappings reflect the database, I have to rewrite my initial migration and delete the migration-history table (there might be a scaffold command for that but I’m doing it in SSMS) and call update-database again.

Using T4Scaffolding to Create DbContext and Repository from Domain Classes

You may already be familiar with the fact that ASP.NET MVC 3 tooling includes a scaffolding option that let’s you point to a domain class and automatically build a Controller, a set of Views and if it does not exist yet, an Entity Framework DbContext class expose that class. The controller CRUD code uses the DbContext to perform it’s operations. Scott’s Guthrie & Hanselman both introduced us to this in blog posts last spring and I’ve certainly demo’d it way too many times.

addctrlr

This is handy but it puts all of the data access code into the controller. In other words, the controller works directly with the DbContext to perform queries & updates (etc).

If you have been paying attention, you may also be aware of the MVCScaffolding NuGet package that adds to these capabilities.

With this installed there are two new templates added to the MVC Add Controller wizard, one which adds a simple repository to the mix.

addctrlr2

MVCScaffolding relies on another NuGet package, T4Scaffolding which contains many of the scaffolding templates used by MVCScaffolding.

I was working on the back end of a new app that may or may not have MVC as it’s front end UI. I had created my domain classes and the next step was to create a DbContext to wrap them as well as repositories. I didn’t want to code those by hand. I wanted that MVC tooling to get some of the grunt work out of the way for me. A little reading helped me learn that I could use the T4Scaffolding directly to get the same DbContext & Repository creation without being part of an MVC app (i.e. no controllers or views created).

Here’s how I worked it out.

In my solution where I already had my DomainClasses project, I added a new project, Data Layer.

I then installed Entity Framework via NuGet into this project:

installef

Notice that NuGet automatically picks up the the latest version of EF, 4.3. I could have used the Package Manager Library UI to install EF, but since I had to the rest in the console window, I’m just doing all of these tasks in the console window.

Next, I install T4Scaffolding.

installt4scaf

Important Pointers!

  • Currently, T4 Scaffolding will install EF 4.1 if you don’t already have EF 4.1 or greater in your project. That’s why I installed EF first, so I can be sure to use the newest version.
  • In the package manager console window, I have my default project pointing to DataLayer. I often forget to specify the project and install NuGet packages into the wrong project. Winking smile

With the package installed, I can now start working towards letting it build out the context & repository for me.

First I need to reference DomainClasses from the DataLayer project

Then I build the solution so that DataLayer can see the DomainClasses for the next step. Smile

Now in the Package Manager Console Window, I’ll use the command to build the repository code. The T4 template that builds the context & repo is called repository. The command is scaffold. I want to run scaffold using the repository template and base the output on my Alias class. I must use Alias’ fully qualified name so that it can be found.

The command is

scaffold repository DomainClasses.Alias

However, I want to specify the name of my context. The default will be DataLayerContext (using the project name). I want it to be TwitterContext.

scaffold repository DomainClasses.Alias -DbContextType:TwitterContext

scaffold1

The result is that a new context & repository class get created in the target project (DataLayer).

There’s some default behavior that is not exactly to my liking and I can use additional parameters as well as modify the T4 template (here’s a post from Steve Sanderson as part of the MVC Scaffolding series that will help with that) but for now I’m happy to just move files around, which I’ll do shortly.

Here’s the context class.

    public class TwitterContext : DbContext
    {
      public DbSet<DomainClasses.Alias> Aliases { get; set; }
    }

Now I’ll add another class from my domain model to the mix: Tweet.

scaffold2

Notice that since TwitterContext already existed, we don’t get a second context class, the template alters the existing one:

    public class TwitterContext : DbContext
    {
      public DbSet<DomainClasses.Alias> Aliases { get; set; }
      public DbSet<DomainClasses.Tweet> Tweets { get; set; }
    }

Finally, I’ll just move things around so they are more to my liking:

solution

I’ve created a Repositories project that has references to EF4.3, DataLayer and DomainClasses.

Now I can use the context & repos as they are or take the generated code (which has saved me a lot of time) and tweak it to my needs. But this has taken a lot of the repetitive typing away from me and started me on a path of success. Smile Happiness.

Thanks to Steve Sanderson & others who worked on this tooling!

(Oh and if you could modify the template to pull in “latest version of EntityFramework.dll”, that would be handy.)

New EF Validations Video on Pluralsight

My newest course is live on Pluralsight.com. This one is all about performing data layer validations in EF 4.1 (4.2, 4.3) using the Validation API that’s part of the DbContext.

Data Layer Validation with Entity Framework 4.1+
Working with the DbContext Validation API

(total 1 hr 54 m)

  • Module 1: Data Layer Validation with Entity Framework DbContext
  • Module 2: Customizing Entity Framework Data Layer Validation
  • Module 3: Integrating Entity Framework Validation with MVC and WCF Data Services

Here’s the entire list of EF courses I’ve done for Pluralsight (so far Smile). I’ve sorted them in the order you might want to watch them in.

Entity Framework (7 courses)

Entity Framework 4.0 By Example
Beginner
[02:18:50]
27 Aug 2010

Entity Framework and Data Models
Intermediate
[01:31:38]
5 Nov 2010

Designer Supported EDM Customization
Intermediate
[02:05:13]
6 Jan 2011

Querying the Entity Framework
Intermediate
[01:22:37]
28 Feb 2011

Entity Framework 4.1 – Code First
Intermediate
[01:58:17]
18 Jun 2011

Entity Framework 4.1 – DbContext Data Access
Intermediate
[01:21:19]
27 Sep 2011

Data Layer Validation with Entity Framework 4.1+
Intermediate
[01:53:49]
8 Feb 2012

Coming soon: Entity Framework 4.3: Code First Migrations and other changes

What do you want to see in Entity Framework?

A friendly reminder about the UserVoice site where you can suggest and vote on features you’d like to see in Entity Framework (http://data.uservoice.com/forums/72025-ado-net-entity-framework-ef-feature-suggestions) or in WCF Data Services (http://data.uservoice.com).

The EF team is wrapping up work on changes to the core EF APIs that will be part of the .NET 4.5 release. We’ve had access to the preview of these new features since June 2011. Here’s an MSDN Magazine article reviewing some of the most prominent features that you’ll find in there: http://msdn.microsoft.com/en-us/magazine/hh394151.aspx.

But EF continues to evolve. The team pays close attention to the suggestions developers make (and vote on) on user voice. In fact, it’s a bit of a popularity contest. If one feature suggestion has hundreds of votes, it will get a lot more attention from the team than a suggestion with only 10 votes.

Here’s a look at hot suggestions.

Top 10 Suggestions Overall

Improved SQL Generation (status=Started) 1204 votes

Batch CUD support (status=Under Review) 981 votes

EF Support for Second Level Cache (status=Under Review) 535 votes

Entity Designer: Speed up & Optimize for using with ~200+ entities 453 votes

Support for multiple databases (status=Under Review) 441 votes

Designer Support GUID as Entity Key  429 votes

Schema Migration a la Rails Migrations (status =Started (Code First Migrations)) 406 votes

TPT (Table-Per-Type) Inheritance Performance (coming in .NET 4.5) 369 votes

Allow filtering for Include extension method (status=Under Review) 319 votes

Allow default date = NOW and provider.MinDate in CSDL (my suggestion Smile) 216 votes
(this isn’t as much of a problem now with code first, but for EDMX it’s still an issue)

There are currently 276 overall. Some have been completed. Some may be duplicates. But if there are features you care about getting into EF, take a look at the list and vote and *promote* the ones you care about.

Oracle Releases Provider with EF 4 Support (but not really EF 4.1 and EF4.2)

Oracle has finally released it’s version of ODP.NET that supports Entity Framework for use in production environments.

This is great news for many developers! 

And they report that this release supports EF 4.1 and EF 4.2. That should mean Code First and DbContext but thanks to Frans Bouma sending me the limitations doc (that is part of the installation ) we can see that they aren’t! Too bad.

7. ODP.NET 11.2.0.3 does not support Code First nor the DbContext APIs.

So, that means “not really EF 4.1 and EF 4.2 support” in my opinion. But they told us already that Code First & DbContext weren’t going to make it in. So it was just the momentary surprise of misunderstanding their meaning about EF4.1 & 4.2 support that deflated me a bit. 😉

Howver…I imagine Code First and DbContext are still leading edge for the types of big corporations that use Oracle. The core EF 4 support is a big deal.

I’ll follow up with Oracle for clarification.

More from Oracle (including downloads) here: http://www.oracle.com/technetwork/topics/dotnet/whatsnew/index.html

Note: They’ve updated the what’s new page to say "Code First is not supported." That’s helpful. Be sure that even thought it’s not mentioned there, EF 4.1/4.2 DbContext API is not supported either.

EMail Q&A: What about Entity SQL and why direct DB access?

I received this pair of questions about EF in email and thought I would share the questions and my reply.

Q:

I’ve read some articles about Code First, I’ve tried to do some samples and I’m going to read you last book.

I’m surprised about 2 issues. The first is that it’s possible to write native SQL code (with context.Database.ExecuteSqlCommand and context.Database.SqlQuery methods) and the second is that I’ve not been able to use EntitySQL or Query Builder Methods.

Now, the idea of EF was create a model, operate with objects and use a common language (EntitSql) to interact with "some" databases. But now? We are coming back? Yes, we can use Linq2Entities and SaveChanges but I don’t understand the philosophy of these opened doors upon the database.

I’m wrong?

What’s your idea?

My A:

Dbcontext doesn’t replace ObjectContext. It just makes it easier for devs to access the most common things.

You can still use ObjectContext. You can even use DbContext and access ObjectContext functions when you need to.

Entity SQL was created before the data team knew about LINQ. You can express most queries with LINQ and it works with any EF-compliant database provider, just as ESQL does.

The direct database access command features are a "backdoor" that devs asked for. It should be used only on rare occasions when you need to access something in the database that is not part of the model.