It was working fine with the Azure AD SSO. ( I was able to use MS account to sign in both jira and confluence)
After I checked the Enforce SSO on the default policy(everyone applies to this policy!),
No one is able to sign in since the SSO process return an http error 404.