Scheduled long running tasks with XAF

Today I will detail about implementing Scheduled Long running tasks in an abstract and reusable way.

The Requirement

Usually, I have a business objects that store time depended data e.g. Exceptions, Logs etc. There is a need to periodically clean up those tables.

The parameters

First we need to identify the problem parameters so we can create a model interface. Parameterizing the problem in a model interface is very useful, because the end user can switch it off if something goes wrong.

Parameters will be: the business object type which we want to clear its records, the criterion to filter the objects and the time to execute this action. The interface along with its registration for this is bellow.

using System;
using System.ComponentModel;
using DevExpress.ExpressApp;
using DevExpress.ExpressApp.Model;
using DevExpress.ExpressApp.Model.Core;

namespace PurgingRule.Module.Controllers{
    //rules container
    public interface IModelClassPurgingRules:IModelClass{
        IModelPurgingRules PurgingRules{ get; }
    }

    [ModelNodesGenerator(typeof (PurgingRulesModelNodesGenerator))] 
    public interface IModelPurgingRules : IModelNode, IModelList<IModelPurgingRule>{
    }

    public interface IModelPurgingRule:IModelNode{
        string Criteria{ get; set; }
        bool ExecuteOnApplicationStart{ get; set; }
        TimeSpan TimeSpan{ get; set; }
        [DefaultValue(500)]
        int ChunkSize{ get; set; }
    }

    //will help us generated more rules if needed
    public class PurgingRulesModelNodesGenerator:ModelNodesGeneratorBase{
        protected override void GenerateNodesCore(ModelNode node){
            
        }
    }

    public class PurgingController:Controller,IModelExtender{

        //model interface registration
        public void ExtendModelInterfaces(ModelInterfaceExtenders extenders){
            extenders.Add<IModelClass,IModelClassPurgingRules>();
        }
    }
}

Now, it is possible to use the Model Editor to configure in which object we want to apply our rules.

That's nice!

but I like to use an Editor to write the Criteria and need to add a few more bits for this to happen. For this we need to use the CriteriaOptions and Editor attributes as shown:

    public interface IModelPurgingRule:IModelNode{
        [Editor("DevExpress.ExpressApp.Win.Core.ModelEditor.CriteriaModelEditorControl, DevExpress.ExpressApp.Win" + XafAssemblyInfo.VersionSuffix + XafAssemblyInfo.AssemblyNamePostfix, typeof(UITypeEditor))]
        [CriteriaOptions("TypeInfo")]
        string Criteria{ get; set; }
        [Browsable(false)]
        [ModelValueCalculator("((IModelClass) Parent.Parent).TypeInfo")]
        ITypeInfo TypeInfo { get; }

Now it should be straight forward to construct any criterion with the help of this build-in editor.

Making Thread Safe Database calls

Working in a multi threaded environment (long running tasks) is build-in in XAF's XPO ORM so we go for it. Just make sure you enable the switch as bellow.

        protected override void CreateDefaultObjectSpaceProvider(CreateCustomObjectSpaceProviderEventArgs args) {
            var threadSafe = true //enable threadsafe;
            args.ObjectSpaceProviders.Add(new XPObjectSpaceProvider(XPObjectSpaceProvider.GetDataStoreProvider(args.ConnectionString, args.Connection, true), threadSafe));
            args.ObjectSpaceProviders.Add(new NonPersistentObjectSpaceProvider(TypesInfo, null));
        }

Scheduling the long running tasks

We will use the StartNewPeriodic extension method and will follow the next steps in our code.

1. Execute once at application startup .
2. Collect the model rules to execute.
3. Calculate the next execution time. For this we need to store the last execution time so I used the RuleScheduleStorage BO class found at the end of the next snippet
4. Periodically schedule parallel calls to the PurgeObjects method for each rule.

    public class PurgingController:Controller,IModelExtender{
        protected override void OnFrameAssigned(){
            base.OnFrameAssigned();
            //1.execute once at application startup .
            if (Frame.Context == TemplateContext.ApplicationWindow){
                //2.collect the model rules to execute.
                var purgingRules = Application.Model.BOModel.Cast<IModelClassPurgingRules>().SelectMany(rules => rules.PurgingRules ).ToArray();
                IEnumerable<(IModelPurgingRule rule, DateTime executed)> ruleExecutionTimes;
                //create an objectspace to query the last execution time from the database
                using (var objectSpace = Application.CreateObjectSpace(typeof(RuleScheduleStorage))){

                    DeleteObsoleteRules(objectSpace, purgingRules);
                    var rulesToSchedule = purgingRules.Where(rule =>rule.Interval!=TimeSpan.MinValue);
                    //get an enumerable of (IModelPurgingRule rule, DateTime executed)
                    ruleExecutionTimes = CalculateExecutionTimes(rulesToSchedule, objectSpace);
                }

                foreach (var ruleExecutionTime in ruleExecutionTimes){
                    var timeSinceLastExecution = DateTime.Now.Subtract(ruleExecutionTime.executed);
                    //calculate if periodic task should start with a delay based time passed since last execution
                    int delay=timeSinceLastExecution<ruleExecutionTime.rule.Interval?(int) ruleExecutionTime.rule.Interval.Subtract(timeSinceLastExecution).TotalMilliseconds:0;
                    //starts the task periodically
                    Task.Factory.StartNewPeriodic(() => PurgeObjects(ruleExecutionTime.rule),
                        interval: (int) ruleExecutionTime.rule.Interval.TotalMilliseconds, delay: delay);
                }
            }
        }

        private static IEnumerable<(IModelPurgingRule rule, DateTime executed)> CalculateExecutionTimes(IEnumerable<IModelPurgingRule> rulesToSchedule, IObjectSpace objectSpace){
            return rulesToSchedule.Select(rule => {
                var ruleScheduleStorage = objectSpace.GetObjectsQuery<RuleScheduleStorage>()
                                              .FirstOrDefault(storage =>storage.RuleScheduleType == RuleScheduleType.Purging && storage.RuleId ==((ModelNode) rule).Id);
                return (rule:rule,executed:ruleScheduleStorage?.Executed ?? DateTime.MinValue);
            });
        }

        private void DeleteObsoleteRules(IObjectSpace objectSpace, IModelPurgingRule[] purgingRules){
            var ids = purgingRules.Cast<ModelNode>().Select(node => node.Id).ToArray();
            var rulesToDelete = objectSpace.GetObjectsQuery<RuleScheduleStorage>().Where(storage =>
                storage.RuleScheduleType == RuleScheduleType.Purging && !ids.Contains(storage.RuleId)).ToArray();
            objectSpace.Delete(rulesToDelete);
            objectSpace.CommitChanges();
        }

You might wonder why I inherit from a controller and used the OnFrameAssigned method as start signal and not simply write my code at Application.SetupComplete.Event. The reason for this is that I like to keep my implementation in separate files and not pollute the Module.cs.

The Long Running PurgeObjects method:

1. Stores last execution time in the database.
2. Deletes filtered objects in chunks.

        private void PurgeObjects(IModelPurgingRule purgingRule){
            try{
                Tracing.Tracer.LogVerboseText($"Purging {purgingRule}");
                var objectsCount = 0;
                using (var objectSpace = Application.CreateObjectSpace(purgingRule.TypeInfo.Type)){
                    StoreExecutionTime(purgingRule, objectSpace);
                    var criteriaOperator = objectSpace.ParseCriteria(purgingRule.Criteria);
                    var objects = objectSpace.GetObjects(purgingRule.TypeInfo.Type, criteriaOperator);
                    objectSpace.SetTopReturnedObjectsCount(objects, purgingRule.ChunkSize);
                    while (objects.Count > 0){
                        objectsCount += objects.Count;
                        objectSpace.Delete(objects);
                        objectSpace.CommitChanges();
                        objectSpace.ReloadCollection(objects);
                    }                    
                }
                Tracing.Tracer.LogVerboseText($"Purged {purgingRule}-{objectsCount}");
            }
            catch (Exception e){
                Tracing.Tracer.LogError(e);
            }
        }

        private static void StoreExecutionTime(IModelPurgingRule purgingRule, IObjectSpace objectSpace){
            var ruleId = ((ModelNode) purgingRule).Id;
            var ruleScheduleStorage =objectSpace.GetObjectsQuery<RuleScheduleStorage>().FirstOrDefault(storage =>
                    storage.RuleScheduleType == RuleScheduleType.Purging && storage.RuleId == ruleId) ?? objectSpace.CreateObject<RuleScheduleStorage>();
            ruleScheduleStorage.RuleScheduleType = RuleScheduleType.Purging;
            ruleScheduleStorage.RuleId = ((ModelNode) purgingRule).Id;
            ruleScheduleStorage.Executed = DateTime.Now;
            objectSpace.CommitChanges();
        }

Last minute feature

I am happy with the PurgingRules, but I would be more happy if I could invalidate them based on a CSharp expression. For example I want to have some rules on my dev machine only.

The PurgingRules already have a criterion so we could create a Custom Function to evaluate CSharp expressions.

I want to use this code in eXpandFramework which there is a <=.NET4 dependency so for evaluating CSharp expressions I chose compile on the fly + caching the from ExpressionEvaluator.Eval method.

    public class EvaluateExpressionOperator:ICustomFunctionOperator{
        public List<string> Usings=new List<string>();

        public const string OperatorName = "EvaluateExpression";

        public static EvaluateExpressionOperator Instance{ get; } = new EvaluateExpressionOperator();

        public Type ResultType(params Type[] operands){
            return typeof(object);
        }

        public object Evaluate(params object[] operands){
            var csCode = string.Join("",operands);
            var usings = string.Join(Environment.NewLine,Usings);
            var eval = ExpressionEvaluator.Eval(csCode, usings);
            return eval;
        }

        public string Name => OperatorName;
    }

Working with such rich libraries like DevExpress XAF suite you make serious stuff in no time!