I want to add my custom block on listing page[Magento-2] but without any modification on list.phtml or any .phtml file. Is there any possibility by using xml file ?

You can do that if you create new module to override this block: vendor\magento\module-catalog\Block\Product\ListProduct.php

to override this block, you need to create di.xml at app\code\Vendor\Module_Name\etc

di.xml content:

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
    <preference for="Magento\Catalog\Block\Product\ListProduct" type="Vendor\Module_Name\Block\Product\ListProduct" />
</config>

Create new file name ListProduct.php at app\code\Vendor\Module_Name\Block\Product

ListProduct.php content:

namespace Vendor\Module_Name\Block\Product;
class ListProduct extends \Magento\Catalog\Block\Product\ListProduct
{
    public function getProductDetailsHtml(\Magento\Catalog\Model\Product $product)
    {
        $html = $this->getLayout()->createBlock('Magento\Framework\View\Element\Template')->setProduct($product)->setTemplate('Vendor_ModuleName::test.phtml')->toHtml();
        $renderer = $this->getDetailsRenderer($product->getTypeId());
        if ($renderer) {
            $renderer->setProduct($product);
            return $html.$renderer->toHtml();
        }
        return '';
    }
}

You can change block Magento\Framework\View\Element\Template to your block

create test.phtml file at app\code\Vendor\Module_Name\view\frontend\templates