It's tricky, because even the Concurrency exceptions that we're experience don't seem to cause a behavioural issue and isn't the actual problem that concerns me the most. The main issue is the fact that users are occasionally getting logged out, which unfortunately I can't replicate in dev or on prod, except for the case where my network configuration changed, which I don't think is the issue that the user had on prod, given the evidence of the two token requests coming in at exactly the same time compared to the network change meaning the token request never even reached the server.
We did have an issue where this front end application was sending multiple token requests on a single tab, which we resolved prior to this instance of the prod issue, but it's also possible the customer had an old version of the front end (which we've also now put mitigations in place for!)
I think we'll see how things go from now on, and consider extending the lifetime of the token (currently only 5 mins), to further reduce the frequency of this issue. I mainly wanted to hear if there was a similar fix that might work for 7.3.3, but it sounds like there isn't.
Thanks.
It will take a very long time to strip out everything that I would need to strip out to give you the bare bones. Is a screen sharing session not possible?
Unfortunately that won't be possible as it contains a lot of IP etc.
I could do a screen sharing session though?
Hi @maliming, Thanks for getting back. I've been able to replicate the Volo.Abp.Data.AbpDbConcurrencyException issue in dev which is helpful for debugging further (similarly to before - just a bunch of tabs open and each getting tokens at approximately the same time).
Interestingly I don't get the logs to show that the Exception is occurring when I override the methods in DbContext, despite seeing my other MPK SaveChangesAsync logs. I find this very surprising given that I can see the call stack originates from Volo.Abp.EntityFrameworkCore.AbpDbContext.SaveChangesAsync. Here is an extract of the logs + I've emailed you the full auth server logs.
[10:17:35 ERR] The database operation was expected to affect 1 row(s), but actually affected 0 row(s); data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.
Volo.Abp.Data.AbpDbConcurrencyException: The database operation was expected to affect 1 row(s), but actually affected 0 row(s); data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.
---> Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: The database operation was expected to affect 1 row(s), but actually affected 0 row(s); data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.
at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ThrowAggregateUpdateConcurrencyExceptionAsync(RelationalDataReader reader, Int32 commandIndex, Int32 expectedRowsAffected, Int32 rowsAffected, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ConsumeResultSetWithRowsAffectedOnlyAsync(Int32 commandIndex, RelationalDataReader reader, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Update.AffectedCountModificationCommandBatch.ConsumeAsync(RelationalDataReader reader, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.SqlServer.Update.Internal.SqlServerModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(IEnumerable`1 commandBatches, IRelationalConnection connection, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(IEnumerable`1 commandBatches, IRelationalConnection connection, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.ExecuteAsync(IEnumerable`1 commandBatches, IRelationalConnection connection, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
at Volo.Abp.EntityFrameworkCore.AbpDbContext`1.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at Volo.Abp.EntityFrameworkCore.AbpDbContext`1.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
at Volo.Abp.Domain.Repositories.EntityFrameworkCore.EfCoreRepository`2.UpdateAsync(TEntity entity, Boolean autoSave, CancellationToken cancellationToken)
at Castle.DynamicProxy.AsyncInterceptorBase.ProceedAsynchronous[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo)
at Volo.Abp.Castle.DynamicProxy.CastleAbpMethodInvocationAdapterWithReturnValue`1.ProceedAsync()
at Volo.Abp.Uow.UnitOfWorkInterceptor.InterceptAsync(IAbpMethodInvocation invocation)
at Volo.Abp.Castle.DynamicProxy.CastleAsyncAbpInterceptorAdapter`1.InterceptAsync[TResult](IInvocation invocation, IInvocationProceedInfo proceedInfo, Func`3 proceed)
at Volo.Abp.OpenIddict.Tokens.AbpOpenIddictTokenStore.UpdateAsync(OpenIddictTokenModel token, CancellationToken cancellationToken)
Here are my overrides:
[ReplaceDbContext(typeof(IIdentityProDbContext))]
[ReplaceDbContext(typeof(ISaasDbContext))]
[ConnectionStringName("Default")]
public class <redacted>DbContext
: AbpDbContext<<redacted>DbContext>,
IIdentityProDbContext,
ISaasDbContext
{
...
public override async Task<int> SaveChangesOnDbContextAsync(bool acceptAllChangesOnSuccess,
CancellationToken cancellationToken = default)
{
Console.WriteLine("MPK SaveChangesOnDbContextAsync");
try
{
return await base.SaveChangesOnDbContextAsync(acceptAllChangesOnSuccess, cancellationToken);
}
catch (Exception e)
{
Console.WriteLine("MPK SaveChangesOnDbContextAsync Exception e");
if (e is not AbpDbConcurrencyException)
{
throw;
}
Console.WriteLine("MPK SaveChangesOnDbContextAsync AbpDbConcurrencyException e");
if (e.InnerException is DbUpdateConcurrencyException dbUpdateConcurrencyException)
{
Console.WriteLine("MPK SaveChangesOnDbContextAsync e.InnerException is DbUpdateConcurrencyException dbUpdateConcurrencyException");
if (dbUpdateConcurrencyException.Entries.Count > 0)
{
Console.WriteLine("MPK SaveChangesOnDbContextAsync dbUpdateConcurrencyException.Entries.Count > 0");
var sb = new StringBuilder();
sb.AppendLine(dbUpdateConcurrencyException.Entries.Count > 1
? "There are some entries which are not saved due to concurrency exception:"
: "There is an entry which is not saved due to concurrency exception:");
foreach (var entry in dbUpdateConcurrencyException.Entries)
{
sb.AppendLine(entry.ToString());
}
Logger.LogWarning(sb.ToString());
}
foreach (var entry in dbUpdateConcurrencyException.Entries)
{
// Reset the state of the entity to prevents future calls to SaveChangesAsync() from failing.
entry.State = EntityState.Unchanged;
}
}
throw;
}
}
public override async Task<int> SaveChangesAsync(CancellationToken cancellationToken = default)
{
Console.WriteLine("MPK SaveChangesAsync");
try
{
return await base.SaveChangesAsync(cancellationToken);
}
catch (Exception e)
{
Console.WriteLine("MPK SaveChangesAsync Exception e");
if (e is not AbpDbConcurrencyException)
{
throw;
}
Console.WriteLine("MPK SaveChangesAsync AbpDbConcurrencyException e");
if (e.InnerException is DbUpdateConcurrencyException dbUpdateConcurrencyException)
{
Console.WriteLine("MPK SaveChangesAsync e.InnerException is DbUpdateConcurrencyException dbUpdateConcurrencyException");
if (dbUpdateConcurrencyException.Entries.Count > 0)
{
Console.WriteLine("MPK SaveChangesAsync dbUpdateConcurrencyException.Entries.Count > 0");
var sb = new StringBuilder();
sb.AppendLine(dbUpdateConcurrencyException.Entries.Count > 1
? "There are some entries which are not saved due to concurrency exception:"
: "There is an entry which is not saved due to concurrency exception:");
foreach (var entry in dbUpdateConcurrencyException.Entries)
{
sb.AppendLine(entry.ToString());
}
Logger.LogWarning(sb.ToString());
}
foreach (var entry in dbUpdateConcurrencyException.Entries)
{
// Reset the state of the entity to prevents future calls to SaveChangesAsync() from failing.
entry.State = EntityState.Unchanged;
}
}
throw;
}
}
public override async Task<int> SaveChangesAsync(bool acceptAllChangesOnSuccess,
CancellationToken cancellationToken = default)
{
Console.WriteLine("MPK SaveChangesAsync 2");
try
{
return await base.SaveChangesAsync(acceptAllChangesOnSuccess, cancellationToken);
}
catch (Exception e)
{
Console.WriteLine("MPK SaveChangesAsync 2 Exception e");
if (e is not AbpDbConcurrencyException)
{
throw;
}
Console.WriteLine("MPK SaveChangesAsync 2 AbpDbConcurrencyException e");
if (e.InnerException is DbUpdateConcurrencyException dbUpdateConcurrencyException)
{
Console.WriteLine("MPK SaveChangesAsync 2e.InnerException is DbUpdateConcurrencyException dbUpdateConcurrencyException");
if (dbUpdateConcurrencyException.Entries.Count > 0)
{
Console.WriteLine("MPK SaveChangesAsync 2 dbUpdateConcurrencyException.Entries.Count > 0");
var sb = new StringBuilder();
sb.AppendLine(dbUpdateConcurrencyException.Entries.Count > 1
? "There are some entries which are not saved due to concurrency exception:"
: "There is an entry which is not saved due to concurrency exception:");
foreach (var entry in dbUpdateConcurrencyException.Entries)
{
sb.AppendLine(entry.ToString());
}
Logger.LogWarning(sb.ToString());
}
foreach (var entry in dbUpdateConcurrencyException.Entries)
{
// Reset the state of the entity to prevents future calls to SaveChangesAsync() from failing.
entry.State = EntityState.Unchanged;
}
}
throw;
}
}
...
Also, it's not making two token requests at the same time (from the same tab), but is making token requests at a similar time in other tabs.
I've sent you the full logs. Cheers.
The AI response doesn't really provide anything new for me.
Thanks. That is a very manual way of doing it, but perhaps the best way. We have since decided to pivot on our approach to one that we prefer that does not require OTP so I haven't confirmed whether this worked for us.
Morning. Any suggestions on this please?
After further investigation, immediately after the loginUsingGrant, this.authService.isAuthenticated returns true and the access_token that was returned is added to the local storage, alongside access_token_stored_at and expires_at.
I can navigate to the user's dashboard (which has an auth guard on it), and in there it also finds this.authService.isAuthenticated returns true. But if I make a query to the public backend, the backend throws Unauthorized, and if I manually refresh the page, this.authService.isAuthenticated is false and the access_token is suddenly removed from local storage.
I can see the Authorization header is set to Bearer XXX from the access_token.
Any tips?
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Authentication.OpenIdConnect;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.DependencyInjection;
using OpenIddict.Abstractions;
using OpenIddict.Server.AspNetCore;
using Volo.Abp.Domain.Entities;
using Volo.Abp.Identity;
using Volo.Abp.Identity.AspNetCore;
using Volo.Abp.OpenIddict.ExtensionGrantTypes;
using IdentityUser = Volo.Abp.Identity.IdentityUser;
using SignInResult = Microsoft.AspNetCore.Mvc.SignInResult;
namespace MyCompany.MyProject.PasswordlessLogin;
/// <summary>
/// Inspired by https://abp.io/community/articles/how-to-add-a-custom-grant-type-in-openiddict.-6v0df94z
/// </summary>
public class PasswordlessExtensionGrant : ITokenExtensionGrant
{
public const string ExtensionGrantName = "PasswordlessAuth";
private readonly IdentityUserManager _userManager;
private readonly AbpSignInManager _signInManager;
public PasswordlessExtensionGrant(IdentityUserManager userManager, AbpSignInManager signInManager)
{
_userManager = userManager;
_signInManager = signInManager;
}
public async Task<IActionResult> HandleAsync(ExtensionGrantContext context)
{
var email = context.Request.GetParameter("email")?.ToString();
var token = context.Request.GetParameter("token")?.ToString();
var user = await _userManager.FindByEmailAsync(email);
if (user is null)
{
throw new EntityNotFoundException(typeof(IdentityUser));
}
var isValid = await _userManager.VerifyUserTokenAsync(user, tokenProvider: "PasswordlessLoginProvider",
purpose: "passwordless-auth", token);
if (!isValid)
{
throw new UnauthorizedAccessException("The token " + token + " is not valid for the user " +
email);
}
await _userManager.UpdateSecurityStampAsync(user);
var principal = await _signInManager.CreateUserPrincipalAsync(user);
principal.SetScopes(principal.GetScopes());
principal.SetResources(await GetResourcesAsync(context, principal.GetScopes()));
return new SignInResult(OpenIddictServerAspNetCoreDefaults.AuthenticationScheme, principal,
new OpenIdConnectChallengeProperties()
{
AllowRefresh = true, // doesn't seem to return a refresh token
IsPersistent = true,
});
}
public string Name => ExtensionGrantName;
private static async Task<IEnumerable<string>> GetResourcesAsync(ExtensionGrantContext context,
ImmutableArray<string> scopes)
{
var resources = new List<string>();
if (!scopes.Any())
{
return resources;
}
await foreach (var resource in context.HttpContext.RequestServices.GetRequiredService<IOpenIddictScopeManager>()
.ListResourcesAsync(scopes))
{
resources.Add(resource);
}
return resources;
}
}
I've had some success. I've been able to write a PasswordlessExtensionGrant : ITokenExtensionGrant that takes email and token request params, verifies the token against the user and calls:
var principal = await _signInManager.CreateUserPrincipalAsync(user);
return new SignInResult(OpenIddictServerAspNetCoreDefaults.AuthenticationScheme, principal);
This actually returns an access token when I call from the front end:
let result = await this.authService.loginUsingGrant('passwordless-auth2', {token: token, email: email},);
{
"access_token": "...",
"token_type": "Bearer",
"expires_in": 299
}
The problem I'm left with is how I do use the access_token in the response to actually login the user in the angular app? When using a password login, it saves the access_token in the local storage (perhaps amongst other things). I haven't been able to see where in abp code it does that, to be able to mimic something similar.
My second concern is that I don't have a refresh_token, so it won't be able to periodically gain new access_tokens.
