staff line detection and removal

[Kumar Shi]

Good day,

I am trying to find a way to remove staff lines from music score for further processing. I am working on OMR (optical music recognition). I am not sure if it is possible with labview but there are some similiar projects using opencv and python. but my work is based on labview. So i was wondering of i can actually use anything to get it done. 

 

I know the steps for staves detection using C++.

1. line detection (using histogram profiling or hough transform)

2. filter the line from original image

 

my problem is that

1. I can use hough transform to detect line but theoritically it won't provide me with enough pixels to remove the whole staff but only a part. so I am wondering if someone has used histogram profiling to detect lines?

2. how to filter the line from original image?

3. any way to import  opencv to labview? ( problem remains because i am using 64 bit system and can't call 32 bit .dll, really stuck on this one.)

 

Thank you so much for any suggestions and help.

[ShaunR]

Off the top of my head.......

 

I would probably do a find edges to locate the ordinates of the lines then draw the lines in the background colour (erasing them) then fill in the bits that were erased that went through the notes with a function like Fill Hole.

[ensegre]

As for a labview wrapper to opencv, I'm only aware of this one (once downloaded but never tried). I'ts all: commercial, closed-source, only windows (and probably only x86), bound to an old opencv version. Given the complexity of opencv though, I think any full scale interface to it would be a major project.

 

I'm not familiar with OMR, but I have done quite some OCR of historical books. For that I actually relied on an existing OS package, tesseract, which is not even the best performing around, but would never have dreamed to implement OCR from scratch using labview [yes there may be some IMAQ "OCR"; but seriously]. Well maybe I could have used labview for just some routinary image rectification and preprocessing task, but it turns there are better ready made tools around, e.g. scantailor. I don't know where you stand in this respect, but if there is some decent OMR around I would stick to it and at best call it from labview if I really had to.

 

Out of curiosity because I don't know OMR workings: does removing the staffs in preprocessing really help the recognition, not complicate it? Well ok, the rhythm of your music is weird, I don't see any two bars with the same duration...

[Kumar Shi]

 

"Out of curiosity because I don't know OMR workings: does removing the staffs in preprocessing really help the recognition, not complicate it?"

 

well, yes removing staffs could help in terms, I can define the region of interest and do template matching to find the duration of the note, e;g, whole note, half note, one fouth note, etc. as for the staff i can get the location of staff by hough transform or line detect on y-axis. then I can compare the location of note on y-axis to find which note is it. e.g, C, C#, D, D#...etc.

but my first problem still remains. what method other than hough transform could be used to find and erase the staffs.

 

 

Edited February 14, 2016 by Kumar Shi

[ensegre]

You might get somewhere by some simpler method, for instance

 

1. compute the marginal pixel sum along verticals
2. divide the image in horizontal stripes, cutting where 1. is zero -> this isolates pentagrams
3. for each of these stripes, compute the horizontal sum
4. a candidate staff is defined in terms of thresholds: the marginal sum of black pixels must be high enough and wide enough (staffs are thicker than note stems)
5. check in each of these locations that [a slight morphological dilation of] the black pixels is exactly five lines high, and pass/fail

which of course assumes that the score has been properly oriented in preprocessing. And still imperfections on the scanned image may fool a simple detection approach; for example the semibischroma D at bar 3 might provide a false staff positive.

 

All together I think complex pattern detection is an art. If there is any attempt of OCM out there, it has really be smarter than any simple scheme like I could think of.

[Kumar Shi]

i agree with you suggestion, i would give it a try. i am aware of complexity of this project. at beginning i am going to implement it on scores downloaded as pdf then convert to bmp files and then do the processing to avoid any complications that cou;d arise from misalignment. thank you for your suggestion. 

[ShaunR]

Used Gradient Morphology instead of the fill holes but not bad for a 5 minute hack, I think.

Edited February 15, 2016 by ShaunR