There are several possible causes for such behavior of Little Snitch.
First of all you really need to make sure that you have set the lifetime for the rule to “Forever“ when creating it (as opposed to “Once“, or “Until Quit“ etc.)
Then it’s also necessary that the path of the application (where it resides on your disk) remains unchanged. If you move an app to a different folder, the rules you created earlier for this app will cease to match.
Unfortunately the path of an application can also change without your explicit doing. If the path displayed in Little Snitch’s Network Alert contains parts like /private/var, it’s likely that the issue is related to the “Gatekeeper” functionality of macOS. See this blog post for further details.