Can we define a Private Constructor

Yes, In static classes.

A private constructor is a special instance constructor. It is commonly used in classes that contain static members only. If a class has one or more private constructors and no public constructors, then other classes (except nested classes) are not allowed to create instances of this class. For example:

class NLog

{

    // Private Constructor:

    private NLog() { }

    public static double e = System.Math.E;

}

The declaration of the empty constructor prevents the automatic generation of a default constructor. Note that if you don't use an access modifier with the constructor it will still be private by default. However, the private modifier is usually used explicitly to make it clear that the class cannot be instantiated.
Private constructors are used to prevent the creation of instances of a class when there are no instance fields or methods, such as the math class, or when a method is called to obtain an instance of a class. If all the methods in the class are static, consider making the entire class static.