Its really easy to debug Silverlight applications in SharePoint 2010. Here are steps:
Step1: Right click in the "SLwithWP" >> Properties >> Click on "SharePoint" tab >> Check the "Enable Silverlight debugging".
Step2: Make breakpoints in the code for Silverlight code
Step3: Press F5 button. I always prefer Firefox. A new Firefox's new window comes up. Enter login credentials. Go to SharePoint 2010 site >> Edit >> Insert >> Web Part >> Under categories, choose "Media and Content" >> Silverlight Web Part >> Add >> Navigate to the XAP file which is under Videos, fill in the URL property: click OK button and our web part has been created.
Step4: Click on "Load" button. Breakpoints will be fried up.
Reference:Interview with Mike Morton about implementing a Silverlight SharePoint WebPart with Visual Studio 2010