Using OpenCV in Android studio to paint the wall in an image

Learn about Android Camera, OpenCV, and storing images in this Android Studio Tutorial.

Version:-

Android Studio 3.x, kotlin 1.3.x

This is a post by Tushar Gusain, an Android and iOS developer.

A good paint job beautifies and adds character and personality to your home. Just for that reason it’s important to take sufficient time planning which colours, shades, and paint styles you want to decorate your home with.

Whenever you buy a new house, whitewash your home, or just want to repaint it you are stuck at deciding which colour will suit best for the walls. Making an app that’ll help you visualise those walls in different colours will come in handy.

We can do this by taking a photo of one or more walls with your phone and clicking the wall to paint them in the chosen colour or texture. Painting the wall, i.e. image processing part can be done with the help of OpenCV library.

Keep reading to know how simple it is to integrate OpenCV in your app.

About OpenCV

OpenCV (Open Source Computer Vision Library) is a library of programming functions mainly aimed at real-time computer vision. Originally developed by Intel, it was later supported by Willow Garage then Itseez (which was later acquired by Intel).

It is an open source library with quite a number of useful Algorithms and API which can be used for image processing. It is mainly used to do all the operation related to Images, like:

1. Reading and Writing Images.

2. Detecting faces and its features.

3. Detecting shapes in an image.

4. Text recognition.

5. Modifying image quality and colours.

6. Developing Augmented reality apps.

Etc…..

OpenCV supports a wide variety of programming languages such as C++, Python, Java, etc., and is available on different platforms including Windows, Linux, OS X, Android, and iOS. Interfaces for high-speed GPU operations based on CUDA and OpenCL are also under active development.

( To know more about OpenCV follow this link: https://opencv.org/about/ )

Creating your Android Studio project

Open Android Studio, click on Start a new project, then choose an Empty Activity and click Next.

Now, Enter the name of your App in the Name field, then, select Kotlin as your Language and click on Finish.

Integrating the OpenCV module

Step 1: Downloading OpenCV

Download the OpenCV library from here: https://opencv.org/releases/
For this project I am using OpenCV — 3.4.7, after downloading unzip the folder.

Step 2: Importing OpenCV

Now, inside Android Studio , Select File > New > import module,

then, for the source directory select OpenCV-android-sdk(the unzipped folder) > sdk > java.

After this click Next then click Finish. This will make an openCVLibrary347 module for you.

Step 3: Fixing the error

Now if you build the project you’ll get an error, to fix this error:
Inside the openCVLibrary347.build.gradle change the compileSdkVersion

and defaultConfig.minSdkVersion to be same as that of app.build.gradle, then

Step 4: Add OpenCV dependency

To work with the OpenCV Android library, you have to add it to your app module as a dependency. To easily do this click on File > Project Structure, click on the app module.

After navigating to the module, click on the Dependencies tab, click the “+” button select Module Dependency, then select OpenCVLibrary347 and then press OK.
After that click Apply then press OK.

Step 6: Add Native Libraries

Now navigate to the unzipped folder of the OpenCV Android library. Open the sdk > native folder then copy the libs folder inside the native folder and paste it to PaintApp (your project name) > src > main folder.

After that rename libs to jniLibs.

Building the User Interface

Step 1: Add a layout to your activity_main.xml

Inside your activity_main.xml add the following layout

<LinearLayout
 xmlns:android=”http://schemas.android.com/apk/res/android"
 xmlns:tools=”http://schemas.android.com/tools"
 android:layout_width=”match_parent”
 android:layout_height=”match_parent”
 android:orientation=”vertical”
 tools:context=”.MainActivity”>
 <ImageView
 android:id=”@+id/imageFromData”
 android:layout_width=”match_parent”
 android:layout_height=”match_parent”
 android:visibility=”visible”/>
 <LinearLayout
  android:id=”@+id/topLayout”
  android:layout_width=”match_parent”
  android:layout_height=”0dp”
  android:layout_weight=”1"
  android:orientation=”horizontal”
  android:visibility=”gone”>
  <RelativeLayout
   android:layout_height=”match_parent”
   android:layout_width=”0dp”
   android:layout_weight=”1">
   <ImageView
    android:id=”@+id/inputImage”
    android:layout_height=”match_parent”
    android:layout_width=”match_parent”
    android:scaleType=”fitXY” />
   <TextView
    android:layout_width=”wrap_content”
    android:layout_height=”wrap_content”
    android:text=”Input Image”/>
  </RelativeLayout>
  <RelativeLayout
   android:layout_height=”match_parent”
   android:layout_width=”0dp”
   android:layout_weight=”1">
   <ImageView
   android:id=”@+id/greyScaleImage”
   android:layout_height=”match_parent”
   android:layout_width=”match_parent”
   android:scaleType=”fitXY” />
   <TextView
    android:layout_width=”wrap_content”
    android:layout_height=”wrap_content”
    android:text=”Grey Scale Image”/>
  </RelativeLayout>
 </LinearLayout>
 <LinearLayout
  android:id=”@+id/middleLayout”
  android:layout_width=”match_parent”
  android:layout_height=”0dp”
  android:layout_weight=”1"
  android:orientation=”horizontal”
  android:visibility=”gone”>
  <RelativeLayout
   android:layout_height=”match_parent”
   android:layout_width=”0dp”
   android:layout_weight=”1">
   <ImageView
    android:id=”@+id/cannyEdgeImage”
    android:layout_height=”match_parent”
    android:layout_width=”match_parent”
    android:scaleType=”fitXY” />
   <TextView
    android:layout_width=”wrap_content”
    android:layout_height=”wrap_content”
    android:textColor=”#FF0000"
    android:text=”Canny Edge Mask”/>
  </RelativeLayout>
  <RelativeLayout
   android:layout_height=”match_parent”
   android:layout_width=”0dp”
   android:layout_weight=”1">
   <ImageView
    android:id=”@+id/floodFillImage”
    android:layout_height=”match_parent”
    android:layout_width=”match_parent”
    android:scaleType=”fitXY” />
   <TextView
    android:layout_width=”wrap_content”
    android:layout_height=”wrap_content”
    android:text=”Flood Fill Image”/>
  </RelativeLayout>
 </LinearLayout>
 <LinearLayout
  android:id=”@+id/bottomLayout”
  android:layout_width=”match_parent”
  android:layout_height=”0dp”
  android:layout_weight=”1"
  android:orientation=”horizontal”
  android:visibility=”gone”>
  <RelativeLayout
   android:layout_height=”match_parent”
   android:layout_width=”0dp”
   android:layout_weight=”1">
   <ImageView
    android:id=”@+id/HSVImage”
    android:layout_height=”match_parent”
    android:layout_width=”match_parent”
    android:scaleType=”fitXY” />
   <TextView
    android:layout_width=”wrap_content”
    android:layout_height=”wrap_content”
    android:text=”HSV Image”/>
  </RelativeLayout>
  <RelativeLayout
   android:layout_height=”match_parent”
   android:layout_width=”0dp”
   android:layout_weight=”1">
   <ImageView
    android:id=”@+id/outputImage” 
    android:layout_height=”match_parent”
    android:layout_width=”match_parent”
    android:scaleType=”fitXY” />
   <TextView
    android:layout_width=”wrap_content”
    android:layout_height=”wrap_content”
    android:text=”Output Image”/>
  </RelativeLayout>
 </LinearLayout>
</LinearLayout>

Step 2: Add strings

Add the following strings to your strings.xml file

<string name=”menu_open”>Final Image</string>
<string name=”menu_select”>Select target</string>
<string name=”menu_cut”>Process Image</string>
<string name=”menu_photo”>Take Photo</string>
<string name=”menu_gallery”>Open Gallery</string>
<string name=”menu_color”>Choose Color</string>
<string name=”menu_texture”>Choose Texture</string>

Step 3: Add menu

Right click on res folder select New > Android Resource File , Give File name as menu_main and Resource type as menu

Now, add the following lines to your main_menu.xml file

<item android:id=”@+id/action_open_img”
 android:title=”@string/menu_open”
 android:orderInCategory=”1"
 app:showAsAction=”never” />
<item android:id=”@+id/action_process_image”
 android:title=”@string/menu_cut”
 android:orderInCategory=”2"
 app:showAsAction=”never” />
<item android:id=”@+id/action_take_photo”
 android:title=”@string/menu_photo”
 android:orderInCategory=”3"
 app:showAsAction=”never” />
<item android:id=”@+id/action_get_gallery”
 android:title=”@string/menu_gallery”
 android:orderInCategory=”4"
 app:showAsAction=”never” />
<item android:id=”@+id/action_get_color”
 android:title=”@string/menu_color”
 android:orderInCategory=”5"
 app:showAsAction=”never” />
<item android:id=”@+id/action_get_texture”
 android:title=”@string/menu_texture”
 android:orderInCategory=”6"
 app:showAsAction=”never” />

Step 4: Add a texture image

To use as a texture image download any png of a texture from internet, i have used the following image as a texture and named it texture_small_brick_red.jpg .

Coding Permissions and Activity

Step 1: Add permissions

In your AndroidManifest.xml add the following lines

<uses-permission android:name=”android.permission.READ_EXTERNAL_STORAGE”/>
<uses-permission android:name=”android.permission.WRITE_EXTERNAL_STORAGE”/>
<uses-permission android:name=”android.permission.CAMERA”/>
<application
 android:allowBackup=”true”
 android:icon=”@mipmap/ic_launcher”
 android:label=”@string/app_name”
 android:roundIcon=”@mipmap/ic_launcher_round”
 android:supportsRtl=”true”
 android:theme=”@style/AppTheme”>
<activity android:name=”.MainActivity”
  android:hardwareAccelerated=”true”>
  <intent-filter>
   <action android:name=”android.intent.action.MAIN” />
   <action android:name=”android.intent.action.VIEW”/>
   <category android:name=”android.intent.category.LAUNCHER” />
  </intent-filter>
 </activity>
 <provider
  android:name=”androidx.core.content.FileProvider”
  android:authorities=”${applicationId}.provider”
  android:exported=”false”
  android:grantUriPermissions=”true”>
  <meta-data
   android:name=”android.support.FILE_PROVIDER_PATHS”
   android:resource=”@xml/provider_paths”/>
 </provider>
</application>

Step 2: Add the provider_paths.xml file

Right click on res folder select New > Android Resource File , Give File name as provider_paths and Resource type as xml

Now, add the following lines to your provider_paths.xml file

<paths xmlns:android=”http://schemas.android.com/apk/res/android">
 <external-path name=”external_files”
  path=”Android/data/com.example.paintapp/files/Pictures”/>
 </paths>

Step 3: Add dependencies

Add the following dependencies in your App build.gradle

dependencies {
implementation fileTree(dir: ‘libs’, include: [‘*.jar’])
implementation “org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version”
implementation ‘androidx.appcompat:appcompat:1.0.2’
implementation ‘androidx.core:core-ktx:1.0.2’
implementation ‘androidx.constraintlayout:constraintlayout:1.1.3’
testImplementation ‘junit:junit:4.12’
androidTestImplementation ‘androidx.test:runner:1.1.1’
androidTestImplementation ‘androidx.test.espresso:espresso-core:3.1.1’
implementation project(path: ‘:openCVLibrary347’)
implementation ‘com.github.yukuku:ambilwarna:2.0.1’ // color picker
}

Step 4: Adding Property Variables and Models

Now, add the following lines of code to your MainActivity

private enum class LoadImage {
 PICK_FROM_CAMERA,
 PICK_FROM_GALLERY
}
companion object {
 init {
  System.loadLibrary(“opencv_java”)
 }
}
var touchCount = 0
lateinit var tl: Point
lateinit var bitmap: Bitmap
var chosenColor = Color.RED
private lateinit var imageFilePath: String
private var texture = false
private val TAG = MainActivity::class.java.simpleName
private val PERMISSIONS = arrayOf<String>(
 Manifest.permission.READ_EXTERNAL_STORAGE,
 Manifest.permission.WRITE_EXTERNAL_STORAGE,
 Manifest.permission.CAMERA
)

Step 5: Setting up rest of the activity

Override OnCreate Method

Add the following lines inside your onCreate method

override fun onCreate(savedInstanceState: Bundle?) {
 super.onCreate(savedInstanceState)
 setContentView(R.layout.activity_main)
 tl = Point()
 openCamera()
}

This’ll initialize your tl(variable to store your touch point) and open camera as soon as the activity is created.

Override onCreateOptionsMenu Method

Add the these lines to your onCreateOptionsMenu method

override fun onCreateOptionsMenu(menu: Menu?): Boolean {
 menuInflater.inflate(R.menu.menu_main,menu)
 return super.onCreateOptionsMenu(menu)
}

This’ll inflate your menu_main file.

Override onOptionsItemSelected Method

Add the these lines to your onOptionsItemSelected method

override fun onOptionsItemSelected(item: MenuItem): Boolean {
 when(item.itemId) {
  R.id.action_open_img -> {
   showImage()
  }
  R.id.action_process_image -> {
   showResultLayouts()
  }
  R.id.action_take_photo -> {
   openCamera()
  }
  R.id.action_get_gallery -> {
   openGallery()
  }
  R.id.action_get_color -> {
   chooseColor()
  }
  R.id.action_get_texture -> {
   chooseTexture()
  }
 }
 return true
}

This’ll call the respective functions for your menu items when they are selected.

Override onActivityResult Method

Add the these lines to your onActivityResult method

public override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
 super.onActivityResult(requestCode, resultCode, data)
 when (requestCode) {
  LoadImage.PICK_FROM_CAMERA.ordinal -> if (resultCode ==   Activity.RESULT_OK) {
   try {
    imageFromData.setImageURI(Uri.parse(imageFilePath))
    bitmap = imageFromData.drawable.toBitmap()
    bitmap = getResizedBitmap(bitmap,bitmap.width/5,bitmap.height/5)
    showImage()
   } catch (e: IOException) {
    e.printStackTrace()
   }
  }
  LoadImage.PICK_FROM_GALLERY.ordinal -> if (resultCode == Activity.RESULT_OK) {
   loadFromGallery(data)
  }
 }
 imageFromData.setOnTouchListener(object : View.OnTouchListener {
  override fun onTouch(v: View, event: MotionEvent): Boolean {
   if (event.action == MotionEvent.ACTION_DOWN) {
    if (touchCount == 0) {
     tl.x = event.x.toDouble()
     tl.y = event.y.toDouble()
     if(texture) {
      applyTexture(bitmap, tl)
     } else {
      rpPaintHSV(bitmap,tl)
     }
    }
   }
   return true
  } 
 })
}

This’ll load the image taken from your camera/gallery into the imageview and add an setOnTouchListener to it , so you can paint the wall you touched in the image.

Override onRequestPermissionsResult Method

Add the these lines to your onRequestPermissionsResult method

override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<String>, grantResults: IntArray) {
 when (requestCode) {
  LoadImage.PICK_FROM_CAMERA.ordinal -> {
   if (grantResults.isEmpty() || grantResults[0] != PackageManager.PERMISSION_GRANTED) {
   Log.e(TAG, “Permission has been denied by user”)
   } else {
    openCamera()
    Log.e(TAG, “Permission has been granted by user”)
   }
  }
 }
}

This’ll open the camera once you give the permissions in your app.

Storing and loading the image

Step 1: Add helper methods

Add the below helper methods to MainActivity,

private fun showImage(image: Mat, view: ImageView) {
 val mBitmap = Bitmap.createBitmap(image.cols(), image.rows(), Bitmap.Config.ARGB_8888);
 Utils.matToBitmap(image, mBitmap)
 view.setImageBitmap(mBitmap)
 bitmap = mBitmap
 saveImage(bitmap)
}
private fun showResultLayouts() {
 imageFromData.visibility = View.GONE
 topLayout.visibility = View.VISIBLE
 middleLayout.visibility = View.VISIBLE
 bottomLayout.visibility = View.VISIBLE
}
private fun showImage() {
 imageFromData.visibility = View.VISIBLE
 topLayout.visibility = View.GONE
 middleLayout.visibility = View.GONE
 bottomLayout.visibility = View.GONE
 try {
  imageFromData.setImageBitmap(bitmap)
 } catch (e: Exception) {
  Toast.makeText(this@MainActivity, “No image selected”,Toast.LENGTH_SHORT).show()
 }
}
private fun chooseColor() {
 texture = false
 val colorPicker = AmbilWarnaDialog(this@MainActivity, chosenColor,  object: AmbilWarnaDialog.OnAmbilWarnaListener {
  override fun onCancel(dialog: AmbilWarnaDialog) {
 }
  override fun onOk(dialog: AmbilWarnaDialog ,color: Int) {
   chosenColor = color
  }
 })
 colorPicker.show()
}
private fun chooseTexture() {
 texture = true
}
fun getResizedBitmap(bm: Bitmap, newWidth: Int, newHeight: Int): Bitmap {
 val width = bm.getWidth()
 val height = bm.getHeight()
 val scaleWidth = newWidth / width.toFloat()
 val scaleHeight = newHeight / height.toFloat()
 // CREATE A MATRIX FOR THE MANIPULATION
 val matrix = Matrix()
 // RESIZE THE BIT MAP
 matrix.postScale(scaleWidth, scaleHeight)
 // “RECREATE” THE NEW BITMAP
 val resizedBitmap = Bitmap.createBitmap(bm, 0, 0, width, height,  matrix, true)
 return resizedBitmap
}
private fun getTextureImage(): Mat {
 var textureImage = BitmapFactory.decodeResource(getResources(), R.drawable.texture_small_brick_red)
 textureImage = getResizedBitmap(textureImage,bitmap.width,bitmap.height)
 val texture = Mat()
 Utils.bitmapToMat(textureImage,texture)
 Imgproc.cvtColor(texture,texture,Imgproc.COLOR_RGBA2RGB)
 return texture
}

Step 2: Opening Camera and Gallery

Add the functions to open gallery and camera, copy and paste the below functions to do that.

private fun openCamera() {
 if (ActivityCompat.checkSelfPermission(baseContext, Manifest.permission.CAMERA)
!= PackageManager.PERMISSION_GRANTED) {
  ActivityCompat.requestPermissions(this, PERMISSIONS, LoadImage.PICK_FROM_CAMERA.ordinal)
  } else {
  val captureIntent = Intent(MediaStore.ACTION_IMAGE_CAPTURE)
  var photoFile: File? = null
  try {
   photoFile = createImageFile()
  } catch (ex: IOException) {
   // Error occurred while creating the File
  }
  if (photoFile != null) {
   val photoURI =   FileProvider.getUriForFile(this,”com.example.paintapp.provider”, photoFile)
   captureIntent.putExtra(MediaStore.EXTRA_OUTPUT, photoURI)
   startActivityForResult(captureIntent,   LoadImage.PICK_FROM_CAMERA.ordinal)
  }
 }
}
private fun openGallery() {
 val i = Intent(Intent.ACTION_PICK,MediaStore.Images.Media.EXTERNAL_CONTENT_URI)
 startActivityForResult(i, LoadImage.PICK_FROM_GALLERY.ordinal)
}

Step 3: Creating and Saving Image files

Now, add createImageFile() function that’ll create an image file for you and saveImage() that’ll save the image in that file. Copy and paste the code below:

private fun createImageFile(): File {
 val timeStamp = SimpleDateFormat(“yyyyMMdd_HHmmss”, Locale.getDefault()).format( Date())
 val imageFileName = “IMG_” + timeStamp + “_”
 val storageDir = getExternalFilesDir(Environment.DIRECTORY_PICTURES)
 val image = File.createTempFile(
  imageFileName, /* prefix */
  “.jpg”, /* suffix */
  storageDir /* directory */
 )
 imageFilePath = image.getAbsolutePath()
 return image;
}
private fun saveImage(image: Bitmap) {
 val pictureFile = createImageFile()
 if (pictureFile == null) {
  Log.e(TAG, “Error creating media file, check storage permissions: “)
  return
 }
 try {
  val fos = FileOutputStream(pictureFile);
  image.compress(Bitmap.CompressFormat.PNG, 90, fos)
  fos.close()
 } catch (e: FileNotFoundException) {
  Log.e(TAG, “File not found: “ + e.message)
 } catch (e: IOException) {
  Log.e(TAG, “Error accessing file: “ + e.message)
 }
}

Step 4: Loading Image from Gallery

Add the below function to load the image from gallery:

private fun loadFromGallery(data:Intent?) {
 val selectedImage = data?.data
 val filePathColumn: Array<String> = arrayOf(MediaStore.Images.Media.DATA)
 val cursor = getContentResolver().query(selectedImage!!,filePathColumn, null, null, null)
 cursor?.moveToFirst()
 val columnIndex = cursor?.getColumnIndex(filePathColumn[0])
 val picturePath = cursor?.getString(columnIndex!!)
 cursor?.close()
 bitmap = BitmapFactory.decodeFile(picturePath)
 bitmap = getResizedBitmap(bitmap,bitmap.width/5,bitmap.height/5)
 showImage()
}

Painting the wall

Apply Paint

Painting the wall consist of the following steps:

1. Get the Greyscale and HSV matrix for the given image
2. Get the S-channel matrix from the HSV image
3. Blur the image matrices
4. Do Canny Edge detection on both of them
5. Now, merge (linear/alpha blend) both the canny edge detected images
6. Dilate the resulting image matrix
7. Flood-fill your original RGB image by taking the above resulting image as a mask and “tl(variable that stores the touch coordinates)” as your seed point with your chosen colour
8. Now, again take the HSV matrix of your original image and merge the V-channel matrix into this flood-filled image
9. Finally add merge the resulting image with your original image.
10. Show your image to the user

The below function will do the task for you:

private fun rpPaintHSV(bitmap: Bitmap, p: Point): Mat {
 val cannyMinThres = 30.0
 val ratio = 2.5
 // show intermediate step results
 // grid created here to do that
 showResultLayouts()
 val mRgbMat = Mat()
 Utils.bitmapToMat(bitmap, mRgbMat)
 showImage(mRgbMat,inputImage)
 Imgproc.cvtColor(mRgbMat,mRgbMat,Imgproc.COLOR_RGBA2RGB)
 val mask = Mat(Size(mRgbMat.width()/8.0, mRgbMat.height()/8.0), CvType.CV_8UC1, Scalar(0.0))
 // Imgproc.dilate(mRgbMat, mRgbMat,mask, Point(0.0,0.0), 5)
 val img = Mat()
 mRgbMat.copyTo(img)
 // grayscale
 val mGreyScaleMat = Mat()
 Imgproc.cvtColor(mRgbMat, mGreyScaleMat, Imgproc.COLOR_RGB2GRAY, 3)
 Imgproc.medianBlur(mGreyScaleMat,mGreyScaleMat,3)
 val cannyGreyMat = Mat()
 Imgproc.Canny(mGreyScaleMat, cannyGreyMat, cannyMinThres, cannyMinThres*ratio, 3)
 showImage(cannyGreyMat,greyScaleImage)
 //hsv
 val hsvImage = Mat()
 Imgproc.cvtColor(img,hsvImage,Imgproc.COLOR_RGB2HSV)
 //got the hsv values
 val list = ArrayList<Mat>(3)
 Core.split(hsvImage, list)
 val sChannelMat = Mat()
 Core.merge(listOf(list.get(1)), sChannelMat)
 Imgproc.medianBlur(sChannelMat,sChannelMat,3)
 showImage(sChannelMat,floodFillImage)
 // canny
 val cannyMat = Mat()
 Imgproc.Canny(sChannelMat, cannyMat, cannyMinThres, cannyMinThres*ratio, 3)
 showImage(cannyMat,HSVImage)
 Core.addWeighted(cannyMat,0.5, cannyGreyMat,0.5 ,0.0,cannyMat)
 Imgproc.dilate(cannyMat, cannyMat,mask, Point(0.0,0.0), 5)
 showImage(cannyMat,cannyEdgeImage)
 val displayMetrics = DisplayMetrics()
 windowManager.defaultDisplay.getMetrics(displayMetrics)
 val height = displayMetrics.heightPixels
 val width = displayMetrics.widthPixels
 val seedPoint = Point(p.x*(mRgbMat.width()/width.toDouble()), p.y*(mRgbMat.height()/height.toDouble()))
// Make sure to resize the cannyMat or it'll throw an error
 Imgproc.resize(cannyMat, cannyMat, Size(cannyMat.width() + 2.0, cannyMat.height() + 2.0))
 Imgproc.medianBlur(mRgbMat,mRgbMat,15)
 val floodFillFlag = 8
 Imgproc.floodFill(
  mRgbMat,
  cannyMat,
  seedPoint,
  Scalar(Color.red(chosenColor).toDouble(),Color.green(chosenColor).toDouble(),Color.blue(chosenColor).toDouble()),
  Rect(),
  Scalar(5.0, 5.0, 5.0),
  Scalar(5.0, 5.0, 5.0),
  floodFillFlag
 )
 // showImage(mRgbMat,floodFillImage)
 Imgproc.dilate(mRgbMat, mRgbMat, mask, Point(0.0,0.0), 5)
 //got the hsv of the mask image
 val rgbHsvImage = Mat()
 Imgproc.cvtColor(mRgbMat,rgbHsvImage,Imgproc.COLOR_RGB2HSV)
 val list1 = ArrayList<Mat>(3)
 Core.split(rgbHsvImage, list1)
 //merged the “v” of original image with mRgb mat
 val result = Mat()
 Core.merge(listOf(list1.get(0),list1.get(1),list.get(2)), result)
 // converted to rgb
 Imgproc.cvtColor(result, result, Imgproc.COLOR_HSV2RGB)
 Core.addWeighted(result,0.7, img,0.3 ,0.0,result )
 showImage(result,outputImage)
 return result
}

Apply Texture

Applying texture is similar to applying paint to your image:

1. Follow Steps 1 to 6 (given above) for the original image
2. For the flood-fill part:

a) wall-image: darken the area of the wall by taking the canny edge detected image as the mask.

b) texture image: brighten the area of the wall by taking the canny edge detected image as the mask.

3. Merge the two image matrices by using Bitwise_or operator on the two matrices.

4. Finally add the V-Channel of the original image and show it to the user.

Copy and paste the below method:

private fun applyTexture(bitmap: Bitmap, p: Point) {
 val cannyMinThres = 30.0
 val ratio = 2.5
 // show intermediate step results
 // grid created here to do that
 showResultLayouts()
 val mRgbMat = Mat()
 Utils.bitmapToMat(bitmap, mRgbMat)
 showImage(mRgbMat,inputImage)
 Imgproc.cvtColor(mRgbMat,mRgbMat,Imgproc.COLOR_RGBA2RGB)
 val mask = Mat(Size(mRgbMat.width()/8.0, mRgbMat.height()/8.0), CvType.CV_8UC1, Scalar(0.0))
 // Imgproc.dilate(mRgbMat, mRgbMat,mask, Point(0.0,0.0), 5) 
 val img = Mat()
 mRgbMat.copyTo(img)
 // grayscale
 val mGreyScaleMat = Mat()
 Imgproc.cvtColor(mRgbMat, mGreyScaleMat, Imgproc.COLOR_RGB2GRAY, 3)
 Imgproc.medianBlur(mGreyScaleMat,mGreyScaleMat,3)
 val cannyGreyMat = Mat()
 Imgproc.Canny(mGreyScaleMat, cannyGreyMat, cannyMinThres, cannyMinThres*ratio, 3)
 showImage(cannyGreyMat,greyScaleImage)
 //hsv
 val hsvImage = Mat()
 Imgproc.cvtColor(img,hsvImage,Imgproc.COLOR_RGB2HSV)
 //got the hsv values
 val list = ArrayList<Mat>(3)
 Core.split(hsvImage, list)
 val sChannelMat = Mat()
 Core.merge(listOf(list.get(1)), sChannelMat)
 Imgproc.medianBlur(sChannelMat,sChannelMat,3)
 showImage(sChannelMat,floodFillImage)
 // canny
 val cannyMat = Mat()
 Imgproc.Canny(sChannelMat, cannyMat, cannyMinThres, cannyMinThres*ratio, 3)
 showImage(cannyMat,HSVImage)
 Core.addWeighted(cannyMat,0.5, cannyGreyMat,0.5 ,0.0,cannyMat)
 Imgproc.dilate(cannyMat, cannyMat,mask, Point(0.0,0.0), 5)
 val displayMetrics = DisplayMetrics()
 windowManager.defaultDisplay.getMetrics(displayMetrics)
 val height = displayMetrics.heightPixels
 val width = displayMetrics.widthPixels
 val seedPoint = Point(p.x*(mRgbMat.width()/width.toDouble()), p.y*(mRgbMat.height()/height.toDouble()))
// Make sure to resize the cannyMat or it'll throw an error
 Imgproc.resize(cannyMat, cannyMat, Size(cannyMat.width() + 2.0, cannyMat.height() + 2.0))
 val cannyMat1 = Mat()
 cannyMat.copyTo(cannyMat1)
 val wallMask = Mat(mRgbMat.size(),mRgbMat.type())
 val floodFillFlag = 8
 Imgproc.floodFill(
  wallMask,
  cannyMat,
  seedPoint,
  Scalar(255.0,255.0,255.0),
  Rect(),
  Scalar(5.0, 5.0, 5.0),
  Scalar(5.0, 5.0, 5.0),
  floodFillFlag
 ) 
 showImage(wallMask,greyScaleImage)
 showImage(cannyMat,cannyEdgeImage)
 Imgproc.floodFill(
  mRgbMat,
  cannyMat1,
  seedPoint,
  Scalar(0.0,0.0,0.0),
  Rect(),
  Scalar(5.0, 5.0, 5.0),
  Scalar(5.0, 5.0, 5.0),
  floodFillFlag
 )
 showImage(mRgbMat,HSVImage)
 val texture = getTextureImage() 
 val textureImgMat = Mat()
 Core.bitwise_and(wallMask ,texture,textureImgMat)
 showImage(textureImgMat,floodFillImage)
 val resultImage = Mat()
 Core.bitwise_or(textureImgMat,mRgbMat,resultImage)
 showImage(resultImage,outputImage)
 ////alpha blending
 //got the hsv of the mask image
 val rgbHsvImage = Mat()
 Imgproc.cvtColor(resultImage,rgbHsvImage,Imgproc.COLOR_RGB2HSV)
 val list1 = ArrayList<Mat>(3)
 Core.split(rgbHsvImage, list1)
 //merged the “v” of original image with mRgb mat
 val result = Mat()
 Core.merge(listOf(list1.get(0),list1.get(1),list.get(2)), result)
 // converted to rgb 
 Imgproc.cvtColor(result, result, Imgproc.COLOR_HSV2RGB) 
 Core.addWeighted(result,0.8, img,0.2 ,0.0,result )
 showImage(result,outputImage)
}

Final result

Now, after getting the image and tapping on the wall inside the image the wall should be painted red (red is the default colour, of course you can choose other colours and your texture from the dropdown menu).

Goodbye

Congratulations!!! , you painted the wall inside the image using OpenCV libraries.

And with that my friend, you are done. Great job!

You can find the whole code for this app here : https://github.com/tushar40/Wall-Paint-App-Android

I hope you enjoyed this tutorial. Feel free to post your questions and comments on the forums, I can’t wait for what you guys come up with for your next great app! ;)

This is a post by Tushar Gusain, an Android and iOS developer.