The key contradict of the dilemma of mathematical truth is that Platonist ontology could not comply with empiricist epistemology, which was the boundary Frege maintained all the time. Inherited from Frege’s theories, Neo-Fregean take linguistic analysis as the guide of ontology. They insist that mathematics should be reduced to logic, emphasize especially the importance of abstraction principle (on ground of contextual principle) in introducing numerical singular, which offers a linguistic solution to the dilemma of mathematical truth. However, they had no reasonable justification for the legitimate position of abstraction principle, which led them to a double attitude to the nature of logic. As a result,if Neo-Fregan insist on the truth of first order logic, they have to face the problem that choice axiom could not reconcile with first order logic,or if they insist on abstraction principle, they have to answer the Caesar problem.