Skip to main content

Data seed through JSON files using EF or any ORM

Last time I had demonstrated a concept for Data Seed through any ORM or MongoDB. The idea is to feed the data to DB at first run or based on condition. The advantage is no one has to manage DB manually with SQL script which feels more natural with Code First approach and every developer can have their own copy with this approach.

I am going to extend the same idea by importing data from JSON file. There are few reasons to do through JSON file as we do not want to keep increasing our codes for data import which may end up with  Combined length of user strings used by the program exceeds allowed limit. Try to decrease use of string literals. Also in the case of a huge number of records it does not make sense and with newer development practice JSON feels more natural for data.

First please go to Data seed for the application with EF, MongoDB or any other ORM as I would be extending same items to allow it through JSON file.

It is always a great idea to have a base class for common items. I would be just creating a new function under BaseSeed base class to import JSON file.

It is better to read any file through streams to avoid any memory clog. So, JsonTextReader would be used through Newtonsoft.Json package.

The whole class definition is this.

  public abstract class BaseSeed<TModel>  
     where TModel : class  
   {  
     protected readonly INoSqlRepoManager NoSqlRepositoryManager;  

     protected readonly IUnitOfWork UnitOfWork;  
     protected readonly IRepository<TModel> ModelRepository;  
    
     public BaseSeed(INoSqlRepoManager noSqlRepoManager)  
     {  
       NoSqlRepoManager = noSqlRepoManager;  
     }  
     
     public BaseSeed(IUnitOfWork unitOfWork)  
     {  
       UnitOfWork = unitOfWork;  
     }  
    
     public BaseSeed(IRepository<TModel> modelRepository)  
     {  
       ModelRepository = modelRepository;  
     }  
    
     protected void Save(TModel model, IRepository<TModel> repository)  
     {  
       repository.Create(model);  
       UnitOfWork.Save();  
     }  
       
     public IEnumerable<T> JsonRead<T>(string file)  
     {  
       if (File.Exists(file)) // No JSON Schema check done.  
       {  
         using (var fs = new FileStream(file, FileMode.Open, FileAccess.Read))  
         using (var sr = new StreamReader(fs))  
         using (var reader = new JsonTextReader(sr))  
         {  
           var jsonSerializer = JsonSerializer.Create();  
           while (reader.Read())  
           {  
             if (reader.TokenType == JsonToken.StartObject)  
             {  
               yield return jsonSerializer.Deserialize<T>(reader);  
             }  
           }  
         }  
       }  
     }  
   }  


JsonRead is a function to read a file as enumerable through yield to avoid memory clog.


Mainly this is the only thing we need to read JSON file and import to DB. The few minor change need to be done in SeedDatabase function to pass the application root path to access file.

JSON shall start as an array with objects, an example from CoreLookup model:

 [  
   {  
    "LookupType":1,  
    "LookupValue":"ALABAMA",  
    "LookupIdString":"AL"  
   },  
   {  
    "LookupType":1,  
    "LookupValue":"Alaska",  
    "LookupIdString":"AK"  
   },  
   {  
    "LookupType":1,  
    "LookupValue":"AMERICAN SAMOA",  
    "LookupIdString":"AS"  
   },  
 ]  

Import of data codes

   
  public class LookupSeeder   
    : BaseSeed<CoreLookup>, ISeed   
  {   
   
    public CoreLookupSeederSql(IMyProjectUnitOfWork MyProjectUnitOfWork)   
       : base(MyProjectUnitOfWork)   
    {   
    }   
   
    public void Seed()   
    {   
       if (!MyProjectUnitOfWork.LookupRepository.Any())  
       {  
         MyProjectUnitOfWork.LookupRepository.Create(JsonRead<CoreLookup>("<JSON file location>");  
         MyProjectUnitOfWork.Save();  
       }  
    }    
  }    

Popular posts from this blog

Architecture solution composting Repository Pattern, Unit Of Work, Dependency Injection, Factory Pattern and others

Project architecture is like garden, we plant the things in certain order and eventually they grow in similar manner. If things are planted well then they will all look(work) great and easier to manage. If they grow as cumbersome it would difficult to maintain and with time more problems would be happening in maintenance.

There is no any fixed or known approach to decide project architecture and specially with Agile Methodology. In Agile Methodology, we cannot predict how our end products will look like similarly we cannot say a certain architecture will fit well for entire development lifespan for project. So, the best thing is to modify the architecture as per our application growth. I understand that it sounds good but will be far more problematic with actual development. If it is left as it is then more problems will arise with time. Just think about moving plant vs a full grown tree.

Coming to technical side, In this article, I will be explaining about the various techniques tha…

LDAP with ASP.Net Identity Core in MVC with project.json

Lightweight Directory Access Protocol (LDAP), the name itself explain it. An application protocol used over an IP network to access the distributed directory information service.

The first and foremost thing is to add references for consuming LDAP. This has to be done by adding reference from Global Assembly Cache (GAC) into project.json

"frameworks": { "net461": { "frameworkAssemblies": { "System.DirectoryServices": "4.0.0.0", "System.DirectoryServices.AccountManagement": "4.0.0.0" } } },
These System.DirectoryServices and System.DirectoryServices.AccountManagement references are used to consume LDAP functionality.

It is always better to have an abstraction for irrelevant items in consuming part. For an example, the application does not need to know about PrincipalContext or any other dependent items from those two references to make it extensible. So, we can begin with some bas…

Configuring Ninject, Asp.Net Identity UserManager, DataProtectorTokenProvider with Owin

It can be bit tricky to configure both Ninject and Asp.Net Identity UserManager if some value is expected from DI to configure UserManager. We will look into configuring both and also use OwinContext to get UserManager.

As usual, all configuration need to be done on Startup.cs. It is just a convention but can be used with different name, the important thing is to decorate class with following attribute to make it Owin start-up:

[assembly: OwinStartup(typeof(MyProject.Web.Startup))]
Ninject configuration

Configuring Ninject kernel through method which would be used to register under Owin.

Startup.cs
public IKernel CreateKernel() { var kernel = new StandardKernel(); try { //kernel.Bind<IHttpModule>().To<HttpApplicationInitializationHttpModule>(); // TODO: Put any other injection which are required. return kernel; } catch { kernel.Dispose(); throw; }…