Hello @Arpit Shah
Welcome to the community.
Jira does not natively provide a method to filter for Epics with no issues. There are a couple of potential solutions.
You could look at third party apps, as discussed in this post.
You might use custom fields and automation to keep track of the number of issues linked to an Epic as those links are generated and removed.