I do not see those categories in the Smart Things settings but also do not use that app at all and I only turned it on to see those choice you provided. So those toggle categories may show because of your connected devices. My experience with making sub choices from any category found, would have this displayed as showing that it can be opened as a separate setting. If the category has a toggle, then it will have a line before the toggle switch. Otherwise, the only action that can be done is to toggle the setting. It appears like this.
Try this, just touch and hold any app you want changed for notification sounds until you get an applet popup. Let go and then click on the tiny circle with an "i" in it at the top right of the applet. Now you should be in the properties of the app. Click on notifications/notification categories
Now some apps will have multiple choice to what you want to hear while some only have one sound to make and you can choose from the example I showed here. TYhe choose the sound you want
Now as far a "custom", I have to ask, do you want your own sound or just a choice from what can be picked natively form the device and the installed apps?
