Did you have enable Intra-Bar Generation with LIBB. If not results are flawed from the get-go. Nothing to do with data.
As for advanced bar type, just like MA's, they are garbage to begin with. Price is price. Period. Fantasy works in Hollywood not Wall Street.
I've never used Tradestation but I was assuming theirs probably some setting like you mentioned.
You really need to understand how whatever backtester your using works.
For instance I use Amibroker I generally keep my orders to the open/close as I can reference which came first and delay my closing orders to the next bar. Their are circumstances where you can use other values but you have to make sure their not forward looking bias.
Another thing is when using indicators ( special candles included) as to how their created. Many indicators are forward looking and their historic values will change as new data is introduced. A quick example I can think of would be Zig Zag. Even a simple moving average system can easily become forward looking.