If this is just a typical adult day center, perhaps consider if there is an adult day program for people with dementia or one for people without cognitive issues. If it is for people with cognitive issues, they are likely used to this problem of refusing to come to the program and may be able to help you with some creative ideas.
One thing that sometimes helps people attend is if they feel they are needed there—that they are volunteering rather than attending as a participant. Another thing you might suggest to the program is that everyone wears name tags so that your loved one doesn’t have to remember names.
Finally, approaching your loved one in a relaxed manner about this—rather than having him or her feel forced to go—may really help. Again, making your loved one feel like he or she is needed and there are people there who he or she can help may be the solution.