In this study, fouling mechanism and modelling analysis of synthetic lignocellulose biomass and agricultural palm oil effluent was studied using polyethersulfone (PES) ultrafiltration (UF) 10 kDa membrane. The impact of process variables (transmembrane pressure (TMP), pH and concentration of feed solution) on lignocellulosic flux was analysed using pore blocking model. The feasible approaches on utilising deep learning artificial neural network (ANN) to predict smaller flux datasets are studied. Among the input variables, pH of lignin feed solution has significant control towards flux and lignin rejection coefficient for both lignin and lignocellulosic solution. Alteration in the structure of lignin at different pH conditions contributed in the improvement of lignin rejection coefficient to 0.98 at the feed pH of 9. A maximum steady state flux of 52.03 L/m2h was observed at the lower lignin concentration (0.25 g/L), TMP of 200 kPa and feed pH of 3. At high TMP and concentration, lignin rejection decreased due to enhancement of feed concentration on membrane surface. The mechanistic model exhibited that cake layer phenomena was dominant in both lignin and lignocellulosic solution. The proposed ANN model showed good correlation (R2-1.00) with experimental non-linear flux dynamic data of both lignin and synthetic lignocellulosic solution. In ANN analysis, activation function, algorithm and neuron effect have significant effect in design of accurate model for prediction of small flux datasets. Aerobically-treated palm oil mill filtration analysis also showed that cake layer phenomenon was dominant. A water recovery of 82 % was achieved even at low TMP under short durations.