ࡱ> AC<=B`!3jz?źf4q_؈qC݋|pOFۊ9c_)[Xk_V:EyOOzcJf f^)Pg(Qo~ )DΫ*O5_ %A"k&oJg.y3cyM7ßq*Ңi=ja^ ?K탮bj|З)v)Xv˸hq%Bg )xWu1 }dqo6n4L' ݡ׸ ݠMn*-ScIZd'mH_bӞǪxcg2V zoΓTyc3H#LM<evySOJX.o_%(<0׈e@~}IH@޷@X&=cG;ا7cF= 9/lL\b~T+#1?g3=)Ӹ:1c^cwS9 AZ?͋l %{ XvQׂq>r+|x>SDTp#o -ѧzMWŲ"//'K~e36k MޫU;:ZqH%CVCQ,["l ommvz3aCGi xC~j3ܨqK9Vj? ű맔eǹn8f1ʏX>L/ߚ" z8bJ"sĢt65O O<0 |1E',[yԵ{<3 4'qNK.э]xtŲE b;)jQ2$ΖtHvGp8aw+R~T 3^G`<8ֵ*p`k N JoqD)?lUnbPbbג}|\{)=|\+Y^> ́qFq^sŸ ,Fw Yt|\]Bw%d"ʆ/K2cL  Z0U-ClI0S zI'Z&ϪSԈdm$и*>؞IJCj~*z=B']?LU=]IF7;+/;X+-VؽramXV]rWscmN|y:& *䁯Go| 2 P,_5 .tG5Gݕ_rإU]{WklOc{peZDzFZBｒ \: +X;bX+IT> 8=z,[-JqaMԉ XVHru{x7h)ӜŷKsƴ1-i <IX?C:e3gR[`㻝m|ˆe$cZe qŘ\RǞ6bOg{壷Tp.<9㻶e0s: \:q=X Sْyĵ5:*=a5=TC{`߃WҀ,!u5Mw| y*|^j 9y_gu[H' mò^|i_}pTm[,Pŗ=\MqMf#znS_K 3e%]Z/p !E.E|Zdre?\2sSp!B}ZDzR+PH*B_ǀY[u]> }5|z|4ŗeG@Uщ/XƋ/]Tm('uRJy|4AO5w6U6&f &XvN卉i>TnLL՞r^!L\<~2ϔR>vĘ;eziτexc: 1gg*JrsaTwWsaoFĘ*{FsaaL{W֞R{HV*%ԨҝJm+tloFmV²s*eR4_LNr^"U;΃)<:wON.w#$}kﾡ:XMThL.r>ZJ<__&,C|idXauh.oPĢGN,2" :+.j79y_}_DYgYBbV>43| xe in1æDK{t3jA7z˵g"/.SܰKϞ8њ8>I ^O3:ߨCesFݏ0e\|`Ԅmm&V fV%^d0Z'F_Q=C=ۂh%m2'bٓ!uѬ!05$Qu-w[/q'3ߓXؚ~0Uw-Zci\5C8CR(Y}ʟ7e1c# Ǹwcfp@f2Hy 9JN·9f8N.z^EI$?׎Bhʐ2)ADv$(;jB3ڊK#b4&o#q0Gweh L$>!ߑg^FN9U\t!ߏ,y9gpObx) Kn1dy̩>S:&|T|Ķ)oǔRR=!W1%U|r>TPmMێZ̗b` ZiаO١qGԻ¬XCB4Y|M b gݣϺn'Pw7tԐՌ5s?%\Bml!|mlOmٔؽgKOԠgI2=N97I \8z/z5kOUO)zSiԨgP>xLTERbF:308Ylt`&M1M^i3h|CJGh2"=CËd?9kZY~;ܰr>|gFj+h%,SmF%IzS¸jj :uPQUtӋUPbŽQsDfM55tZ;$Liyn\׼UҼ E1=gHb1559fH]j~ڛSt&9r]^`x3FTIJzSr_Bl%Hneq:4Mfӯ<-Gc^f^?s8j suWw2RBAzHȹO{( 6$svihqex.v6#NF5ݞJ2: vp81)-hs' -;p^$Swx^#5jښ8RC':_KqʕTڇ+tEV|m&O b6:["Iw~x TWC^$ByZi1r}q' Ӧ5O+ZʝTKP|-]UVUuǹ͈v*v֮@y];a(Ϳʍ9sF r;;9A9I;V?P 眺DbNK{i8AX;eƟ@Z;Sn BeG<2tvgtXeikM͢کQuګ1lnLU h9ΈVǤ#ɪ4ΖTh:5p:Ng[K*Hy]{KgCgeh:hl|Vq+lameH{q=014eht#Hm#`!T X! xڍMhAɧREj 6h/V54@MۂbŃ<,\x(xzEzIEqfLȄT0޾7nf &q͖( j2NJ#} 3kXŃ!ZlS#c|0[)ެ-y\bk;VYep _1fDwNPmSo:|1R;׹"nYUKgQc+ɬe><ԟG [甥y"@QQs>{:7lS;D5߮Mrt9OTPOe}[OS6ۺHYAV:캫ξ')~;YAc̺V/n׵zERmi^Nk$kZR:q/)qp/+@wzP'O|:_4'ܰgldp-33D{uʞhD'5MkTܟʨҟbY*>U5Bь6?@*e?vC"95"R3^NN,J!=μ^w!&RʊSqLW/( KAķ dxR(q0E0 !Tg6E_ F#(in   9eEquation Equation.30,Microsoft Equation 3.0fEquation Equation.30,Microsoft Equation 3.0 gEquation Equation.30,Microsoft Equation 3.0!hEquation Equation.30,Microsoft Equation 3.0N/ 0|DTimes New Roman( 0( 0 DArialNew Roman( 0( 0  DWingdingsRoman( 0( 0 0DArial Blackman( 0( 0 "@DVerdanaackman( 0( 0 " C0.  @n?" dd@  @@`` (%   $,+0. / 15>TUWY[\ hkl r vxyhd;/*7o2$3jz??G %8 ʚ;Afm)ʚ;g4.d.d@ 0.ppp@ <4ddddlqt 0\-<4KdKdlqt 0\-<4!d!dlqt 0\-0___PPT10 pp___PPT9f^"( ?  O  =T%@Dichotomization of ICU length of stay based on model calibration$AJ"ICU length of stay (LOS)Important outcome after cardiac surgery Predictive models for identification of high risk patients case load planning and resource allocation \( - -P -_'Main objectiverDevelopment of a predictive model to estimate the risk of long ICU LOS using the method of class probability trees:s: f,m/Data2063 patients (Academic Medical Center, Amsterdam, 1997-2001) preoperative (e.g., age, gender) operative (e.g., surgery type, duration) first 24h ICU stay (e.g., blood pressure, temperature) (122 patients died: 5.2%)> -R  n.Problem of outcome definition\How should we define the outcome  long ICU LOS ? Literature: outcome dichotomized based on threshold values of 2-10 days without motivation or based on simple statistics.$ fs2Objective of this studyfSelection of the threshold value to dichotomize ICU LOS in a structured fashion based on data analysis.gfG`(ApproachDevelopment of tree models for outcomes defined with different threshold values Calculation of the model performance Selection of the best model  " d " d5fff  First resultsv4!Distances between probabilities I""( t3!Distances between probabilities I""( c+!Distances between probabilities I""( i, ALOR distance1 Distance between two probabilities for a given x22P/q1"Distances between probabilities II##(7Property of ALOR: approximate proportional equivalence,8Z$fj-MALOR statistic' Distance measure for all elements in FB(! Procedure of threshold selection 1) define a set of possible threshold values T 2) for all threshold values t in T do a) define the dichotomized outcome Yt using threshold t b) build a predictive model Mt for outcome Yt c) compute DMALOR(Mt , Pt) 3) select threshold value with minimal MALOR statisticHZ -(        9>{@ ?o0Additional resultsS$PTree model for  ICU LOS>5 days or death ))$V&Discussion and conclusions0Class probability trees to identify high risk groups Performance measure should be insensitive to class unbalance when comparing models for different prediction problems:f<f85 j/:;=LU]#^$d%k&l'p(r)u*w+J ` fff33` 3KI3ff` 33ff` /p` 3%*3|` Jy3fff3f` 3ff3̙` 33ff33` DDyq3f` ̙3n` w3ff` }ff>?" dd@,?nKd@ P nA@F`d n?" dd@   @@``PR"   @ ` `2p>> n f x (  x x 6Ԝ #" ``   b*0  x 6@ #" `` `  b*0 T X x "X x N`d#" `P :  x 6d#" `U   x S "UY F 0  x c $"YW F 0   x c $H"YU F 0   x c $" F 0   x S к#" `SV    x S 轥"Y F 0   x c $@"X F 0  x <å #" `  `  T Click to edit Master title style! !$ x 0ť " `p  RClick to edit Master text styles Second level Third level Fourth level Fifth level!     S x 6ʥ #" `^ `  `*0 H x 0޽h ? }ff80___PPT10. 07  Pixel     | (  |T  | " | Th d #"   :  | c $ "9)e   b e  |# "e  | S "ie    | S "9)   | S ,"0    | S D"?e     | S ȥ")    | S "?G    | S  "oG    | S 8"9G   | S d"iA   | S "A?   | 6 #" `` `  `*0  | 6` #" ``   b*0  | 6 #" `` `  b*0  | 0` "P    T Click to edit Master title style! ! | 0| " P   W#Click to edit Master subtitle style$ $H | 0޽h ? }ff80___PPT10. 070 zr (    0<[ 9  [ P*    0[ K h9 [ R*  d  c $ ?Q  [  0[   [ RClick to edit Master text styles Second level Third level Fourth level Fifth level!     S  6[ 4m  [ P*    6h[ 4K hm [ R*  H  0e ? 3380___PPT10.\pp (    0 9   X*   0 K h9  Z*   64 4m   X*   6 4K hm  Z* H  0e ? 3380___PPT10.0'@  ` l(  r  S 4[x<S   [   nAxTUelichtgeel9 _U [VR  l" 3 f TJ   BCDE`FlfY  ~ oL ] I 2 ? y   wPO' 7n^;  a & " : Pq c1 s  n +   R   {{ j6 V@%o0~vGKUN-  ]   u aQ Q C :z 50 3 5 :P C Q ay u4 n/-U}FG~T+0o6 t{ ` P CR : 4 2 2" 3? 4\ 6y 9 < @ D u o j g c[ `5 _ ^ ] _" g sd  QPFteRSduGOO b s g _ ] _H g s i  ! {  'G{ueS8we#UPQ 1d F W" c k m@ k e [G M ; &A   -w] 1MaK'i&MxL c  6 Il Y @`@` l"    BCDEFf))NY.guc2QG?[.m~ {hVC2sf[OE;x3e+R$<%d/\ ( +[7,CP`qyN$`<4LfvX;3Tvy>l\ayXQKFA?%>D>a>?DK-Sf\.Mi  % +)013:6D7O7Y7e6r3(0G]ry`G-{m_P7 yjI$blrw| #18?FKiON6MDm|iUA -3[=lbX|E3!zsmga]XQtL^IHH5GGGIJLuPQU,Y_fnvi?qG&x<QS)jnf]UM<(.@"[w !;UlUX@`@`  M    B~ CDEFf((Q{bn86Vv`*,S[|* "~7lLZ\RkK|D?940,*'& %%(%6'C*P,]0i4t:@GOXalx(*R{q1ZVUUVX\aekqx_,IfX&k)?Msvc-RGFa<y51..15; D N$ Y1 f> tI T ] e l r w z } ~ %~ 7} Jz [   ! 5 H [ k y j W E +0 ; J Xfs}mR8o_{QnD^:L29,$) (+8}F)TdmN^  ztnS8,RznL= ^Q$p||!kW3-?JtSZ\]]]\[YzVuSoOjLgGc>]3[)Z[] `cgjnoR67]4es^I}}HQ{RT@`  N   0[    JMarion Verduijn, Niels Peek, Frans Voorbraak, Evert de Jonge, Bas de Mol ,K;   s *["`m D0 2 H  0޽h ? }ff___PPT10i.}`֞+D=' = @B +  0(  x  c $hx,'  b x  c $)x `p  H  0޽h ? }ff___PPT10i.|e+D=' = @B +}   $$(  $r $ S Ibx  `  b r $ S Jbx `  b H $ 0޽h ? }ff___PPT10i.(`+D=' = @B +  X0(  Xx X c $Vbx  `  b x X c $Wbx ` b H X 0޽h ? }ff___PPT10i.(`+D=' = @B +  T0(  Tx T c $gbx  `  b x T c $hbx ` b H T 0޽h ? }ff___PPT10i.(`+D=' = @B +  P t0(  tx t c $ubx  `  b x t c $vbx ` b H t 0޽h ? }ff___PPT10i.(`+D=' = @B +  0(F(  (x ( c $bx  `  b  ( c $Lbx `p b "p`PpH ( 0޽h ? }ff___PPT10i.(`+D=' = @B +<0  K0 ;/3/@D.(  x  c $dbx  `  b , z  #">2 >z b  <[ ?"` z O0.216 @`  <b ?"`   O0.232 @`  <$>[ ?"`   O0.230 @`  <b ?"`   O0.242 @`  <b ?"`7   O0.274 @`  <b ?"`U 7  O0.280 @`  <b ?"`U  O0.293 @`  <b ?"` O0.348 @`  <b ?"` O0.418 @`  <b ?"` O0.475 @` l <b?O   T  @` k <4b?' O  O0.141 @` j <? '  Q10 days @`  <?OU  T  @`  < ?'OU  O0.245 @`  <!?'U  P5 days @`  <*?OU 7  T  @`  <p3?'U O7  O0.207 @`  <x<?U '7  P6 days @` z <DE?O7   T  @` x <$N?'7 O  O0.182 @` v <W?7 '  P7 days @` q < Y?O   T  @` o <hi?' O  O0.167 @` m <pr? '  P8 days @` h <xt?O   T  @` f <Ԅ?' O  O0.156 @` d <܍? '  P9 days @` Y <?O T  @` W <@?'O O0.548 @` U <H?' P2 days @` P <P?O T  @` N <?'O O0.390 @` L <?' P3 days @`  <?O z T  @`  <?' Oz O0.129 @`  <? 'z Q12 days @`  <?O T  @`  <4?'O O0.292 @`  <<?' P4 days @`(  <?O HBrier score! class probability tree :%  @`  <H?'O z$proportion events $ @`  <?' U threshold   @`ZB ! s *o ?ZB " s *1 ?ZB % s *o ?zz`B & 0o ?`B - 0o ?`B  0o ?`B  0o ?`B  0o ?U `B  0o ?U 7 `B  0o ?7  `B  0o ?  `B  0o ?  `B  0o ? z`B Ϙ 0o ?`B Ә 0o ?`B ט 0o ?U `B ۘ 0o ?U 7 `B ߘ 0o ?7  `B  0o ?  `B  0o ?  `B  0o ? zp  0!v  patients with ICU LOS higher than the threshold value of death ! determined using 10-fold cross-validationm0m H  0޽h ? ̙33___PPT10i.+D=' = @B +Y + ph ))(  x  c $x  `    '+  # #""kml',    <?5L 1  M0.666 @`  <H ?51 L  M0.666 @`  <?5`1  M0.666 @`  <!?51 ` dRelEr4   @`  <H+ ?"`xL +  P0.000108   @`   <44 ?"`x+L  O0.00109 @`   << ?"`x`+ N0.0122 @`   <G ?"`x+` udKL&  @`   <P?-L x  R 0.00000025   @`   <$Y?1 L -  N0.0005 @`  <b?sL 5  N0.0015 @`  <$k?'L s  M0.001 @`  <t?-xL  P0.000025   @`  <x}?1 -L  M0.005 @`  <솦?s5L  M0.015 @`  <x?'sL  L0.01 @`  <옦?-`x N0.0025 @`  <x?1 `- L0.05 @`  <?s`5 L0.15 @`  <?'`s K0.1 @`  <Խ?-x` wdSqEr&  @`  <4?1 -` xdAbsEr&  @`  <Ѧ?s5` tMx&  @`  <Lۦ?'s` tPx&  @`ZB  s *o ?'+ZB  s *1 ?'`+`ZB  s *1 ?'+ZB  s *1 ?'L +L ZB   s *o ?' + `B ! 0o ?''``B " 0o ?++``B # 0o ?'`'`B $ 0o ?+`+`B % 0o ?''L `B & 0o ?++L `B ' 0o ?'L ' `B ( 0o ?+L + ` ) s *"`Y H  0޽h ? }ff___PPT10i.8~@:+D=' = @B + * &` *.x(  xx x c $ľx  `    '+  x# #""kml',   x <?5L 1  M0.666 @` x <?51 L  M0.666 @` x <?5`1  M0.666 @` x <?51 ` dRelEr4   @`  x < ?"`xL +  P0.000108   @`  x <?sL 5  N0.0015 @`  8 <G?'L s  M0.001 @`  8 <I?-xL  P0.000025   @`  8 <8Z?1 -L  M0.005 @`  8 <T?s5L  M0.015 @` 8 <k?'sL  L0.01 @` 8 <Du?-`x N0.0025 @` 8 <}?1 `- L0.05 @` 8 <Ԇ?s`5 L0.15 @` 8 <؏?'`s K0.1 @` 8 <x?-x` wdSqEr&  @` 8 <\?1 -` xdAbsEr&  @` 8 <H?s5` tMx&  @` 8 <D?'s` tPx&  @`ZB 8 s *o ?'+ZB 8 s *1 ?'`+`ZB 8 s *1 ?'+ZB 8 s *1 ?'L +L ZB 8 s *o ?' + `B 8 0o ?''``B 8 0o ?++``B  8 0o ?'`'`B !8 0o ?+`+`B "8 0o ?''L `B #8 0o ?++L `B $8 0o ?'L ' `B %8 0o ?+L + H 8 0޽h ? }ff___PPT10i.8~@:+D=' = @B + & @I(  @x @ c $<̨x  `   ~ @ s *Hƨx w2   @0 TA ? ?xDp   @0 TA ? ?x2  $    @ 0p ` [Absolute Log-Odds Ratio(ZffH @ 0޽h ? }ff___PPT10i.5~P)ߚ+D=' = @B +t  ) 0 -:h(  h~ h s *,٨x `  c   8h #""kml   h <?L   M0.666 @` h <?L  M0.666 @` h <?` M0.666 @` h <D?` dRelEr4   @`  h <,  ?"`L   N0.4060 @`  h <  ?"`L  N0.4105 @`  h < ?"`` N0.4626 @`  h <" ?"`` wdALOR&  @`  h <4, ?"`&L   P0.000108   @` h <85 ?"`&L  O0.00109 @` h <& ?"`&` N0.0122 @` h <G ?"`&` udKL&  @` h <pQ? L &  R 0.00000025   @` h <8Z?L   N0.0005 @` h <pc?L   N0.0015 @` h <l?L   M0.001 @` h <Lu? &L  P0.000025   @` h <p? L  M0.005 @` h <?L  M0.015 @` h <0?L  L0.01 @` h <? `& N0.0025 @` h <0?`  L0.05 @` h <?` L0.15 @` h <0?` K0.1 @` h <p? &` wdSqEr&  @` h <з? ` xdAbsEr&  @` h <\ҫ?` tMx&  @`  h <۫?` tPx&  @`ZB !h s *o ?ZB "h s *1 ?``ZB #h s *1 ?ZB $h s *1 ?L L ZB %h s *o ?  `B &h 0o ?``B 'h 0o ?``B (h 0o ?``B )h 0o ?``B *h 0o ?L `B +h 0o ?L `B ,h 0o ?L `B -h 0o ?L   :h c d0e0e x3"0e`  `   H h 0޽h ? }ff___PPT10i.8~@:+D=' = @B +m ' |H(  Hx H c $P x  `    H s */xw   H0 TA  ? ?xDp   H0 TA !? ?xPN  ! x H 0b[ s    Mean value of the Absolute Log-Odds Ratio (MALOR) quantifies model calibration XS*f ffH H 0޽h ? }ff___PPT10i.5~P)ߚ+D=' = @B +  K0 t(  x  c $(x       S  x><$D0  *K=MH  0޽h ? ̙33B:___PPT10.7+YD' = @B D' = @BA?%,( < +O%,( < +Dt' =%(D' =%(D' =4@BBBB%())))?D' =1:Bvisible*o3>+B#style.visibility<*0%(Dt' =%(D' =%(D' =4@BBBB%())))?D' =1:Bvisible*o3>+B#style.visibility<*0W%(Dt' =%(D' =%(D' =4@BBBB%())))?D' =1:Bvisible*o3>+B#style.visibility<*W%(Dt' =%(D' =%(D' =4@BBBB%())))?D' =1:Bvisible*o3>+B#style.visibility<*%(Dt' =%(D' =%(D' =4@BBBB%())))?D' =1:Bvisible*o3>+B#style.visibility<*%(Dt' =%(D' =%(D' =4@BBBB%())))?D' =1:Bvisible*o3>+B#style.visibility<*%(+H ( K0 @@ \c`E@(  `~ ` s *.x  `   ;< YP c` #"B6 YP  ` <,z[ ?"` r0P O0.181 @` ` <2 ?"` 0r O0.193 @` ` <= ?"` 0  O0.206 @` ` <C ?"` 0  O0.212 @` ` <}[ ?"` 0  O0.232 @`  ` <U ?"`  0  O0.251 @`  ` < _ ?"` & 0  O0.280 @`  ` <4h ?"` C0&  O0.312 @`  ` < q ?"` h0C O0.366 @`  ` <Dz ?"` 0h O0.415 @` ` <t ?"` G0 Y tree ensemble @` ` <t ?"`0 Yr O0.765 @` ` < ?"` r O0.232 @` ` <? r O0.141 @` ` < ? r Q10 days @` ` < ?"`0& Y  O0.468 @` ` <h ?"`&   O0.293 @` ` <?&   O0.245 @` ` < ˬ?&   P5 days @` ` <Ӭ ?"`0 Y  O0.534 @` ` <ܬ ?"`  O0.280 @` ` <?   O0.207 @` ` <,?   P6 days @` ` < ?"`0 Y  O0.575 @` ` < ?"`  O0.274 @` ` < ?   O0.182 @` ` <?   P7 days @` ` < ?"`0 Y  O0.618 @`  ` <t% ?"`  O0.242 @` !` </?   O0.167 @` "` <\)?   P8 days @` #` <0 ?"`0 Y  O0.616 @` $` <$H ?"`  O0.230 @` %` <,Q?   O0.156 @` &` <4Z?   P9 days @` '` <` <5?G U threshold   @`ZB ?` s *o ?YZB @` s *1 ?GYGZB A` s *1 ?YZB B` s *o ?PYP`B C` 0o ?G`B D` 0o ?YYG`B E` 0o ?G`B F` 0o ?h`B G` 0o ?hC`B H` 0o ?C& `B I` 0o ?&  `B J` 0o ?  `B K` 0o ?  `B L` 0o ?  `B M` 0o ?  `B N` 0o ? P`B O` 0o ?YGY`B P` 0o ?YYh`B Q` 0o ?YhYC`B R` 0o ?YCY& `B S` 0o ?Y& Y `B T` 0o ?Y Y `B U` 0o ?Y Y `B V` 0o ?Y Y `B W` 0o ?Y Y `B X` 0o ?Y YPp Y` 0H>9Y  patients with ICU LOS higher than the threshold value of death ! determined using 10-fold cross-validationm0m 2 Z` s *"`F, ,$D02 [` s *"`}F,a,$D02 \` s *"``F,D,$@0H ` 0޽h ? ̙33^V___PPT106.+QHD ' = @B D' = @BA?%,( < +O%,( < +D4' =%(D' =%(D' =4@BBBB%(D' =1:Bvisible*o3>+B#style.visibility<*Z`%(D' =%(Dh' =%(D' =4@BBBB%(D' =1:Bvisible*o3>+B#style.visibility<*\`%(D' =4@BBBB%(D' =1:Bvisible*o3>+B#style.visibility<*[`%(+ #  \(  x  c $$x<     dA xRTfivedays] H  0޽h ? }ff___PPT10i.{(+D=' = @B + $ n(  r  S B[x  `  [   S [x `p<$D0 [ *CDEFH  0޽h ? }ff___PPT10.`O+YD~' = @B D9' = @BA?%,( < +O%,( < +D4' =%(D' =%(D' =4@BBBB%(D' =1:Bvisible*o3>+B#style.visibility<*5%(D4' =%(D' =%(D' =4@BBBB%(D' =1:Bvisible*o3>+B#style.visibility<*5%(+i K0 h`P(    0Y*@ `&Marion Verduijn m.verduijn@amc.uva.nl'0 'H  0޽h ? ̙33___PPT10i.t`+D=' = @B + 0 NF@T(  T^ T S P   @ T c $,`    We show the first results in this table. In the first two column the threshold is shown and the numbers of observations with outcome class long LOS or death. We quantified performance of the class probability trees using the Brier score, shown in the third column. As you can see, the higher the threshold value, the better the model performance (smaller Brier scores). This pattern of decreasing values for the Brier scores is a direct result of the fact that the proportion of events decrease and the prediction problem gets more unbalanced. So the Brier score is sensitive to class unbalance, and therefore not appropriate to quantify model performance to select the threshold value of the best model.H T 0e ? 3380___PPT10.`PX0 5-PX(  X^ X S P   ' X c $$q    We used the MALOR statistic in our procedure for threshold selection. This procedure contains of the following steps. First, define a set of possible threshold values. For each threshold value, define the dichotomized outcome, and build a predictive model for this outcome. Furthermore compute the model calibration using the MALOR statistic. Finally, select the threshold value with a minimal value for the MALOR statisticH X 0e ? 3380___PPT10.ap{E?0 hO(  h^ h S P    h c ${    E1Thank you for listening. Are there any questions?H h 0e ? 3380___PPT10.d峑W"0 g(  d  c $P     s *    QIn our project, we focus on the patient group of cardiac surgery. After the cardiac intervention like bypass and valve operations, almost all patients are sent to the ICU for recovery. The length of stay at the ICU is an important outcome for the health care process of this relatively homogenous patient group within Intensive Care Medicine. The outcome ICU LOS is a proxy of the degree of complication and the severity of postoperative illness. When no complications occur, the patient is discharged from ICU within 24h. Several predictive models have been developed to predict the ICU LOS for cardiac surgical patients to identify high risk patients. Furthermore, the predictions are useful for the ICU management for case load planning and resource allocation, as ICM is expensive and ICU beds are scarce. Good estimation of the probability that patients will be discharged gives the chance to approximate the numbers of beds that will be empty within a certain time. These models have thus mainly their function at the level of patient groups. } H  0e ? 3380___PPT10.l23%0 ~(  ^  S P   bx  c $b   b Ok let s get started. Good morning, my name is Marion Verduijn. In this presentation, I will tell you about our project that is done at the Academic Medical Center in Amsterdam, in collaboration with the Eindhoven Technical University, The Netherlands H  0e ? 3380___PPT10.}Q<p$0 (  X  C P     S     nHere the tree model for the outcome  ICU LOS > 5 days or death is shown. This tree model that can be used in clinical practice is a direct result of applying the presented procedure.H  0e ? 3380___PPT10.C0&0 .(  X  C P     S |    0We used the method of class probability trees for model development. These models are comprehensible for clinicians in describing high risk groups due to their tree structure. However, these models are known to be quite unstable, and tend to benefit substantially from ensemble learning. However the tree ensembles itself are not comprehensible. For that reason, we were willing to give up some performance when building the tree models that are to be used in clinical practice, and used approximation of the true class probabilities of the tree ensemble method when determining calibration of the tree models using the MALOR statistic. Finally, I will note that the insensitiveness to class unbalance is necessary to make a fair comparison of model performance for different prediction problems.H  0e ? 3380___PPT10.HV6+0 jb<(  <d < c $P   V < s *    In the additional columns is shown how the distances are valued by well-known distance measures such as the squared error and the Kullback-Leibler distances. As you can see that the values of these measures become steadily smaller as the probabilities get smaller. So, when determining the performance of predictive models, these distance measures will always value the model for the most imbalanced problem to be best calibrated.H < 0e ? 3380___PPT10.o,0 ldD(  D^ D S P   ^ D c $    We developed a new statistic to quantify the distance between two probabilities. It is called the ALOR distance as it quantifies the distance between two probabilities for a given x in terms of the absolute log odds ratio.H D 0e ? 3380___PPT10.@ e-0 L(  L^ L S P    L c $    The MALOR statistic quantified the ALOR distances for all elements in the features space. It is the mean value of the absolute log-odds ratio.H L 0e ? 3380___PPT10.@4r00 5- d(  dd d c $P   ! d s *    The MALOR statistic is shown in the fifth column. It takes relatively low values at low thresholds, and reaches its minimal value at a threshold of 5 days. The value of the MALOR statistic for this threshold does not significantly differ to the value of the MALOR statistic for two and three days. So, these threshold values are also found to be good candidates to dichotomize ICU LOS.H d 0e ? 3380___PPT10.`PX710 @ lG(  ld l c $P    l s *l    1The ALOR distance for the pairs of probabilities is approximately equal when reducing both probabilities using the same factor, and this equivalence increases as the probabilities become smaller. So, the ALOR distance has the important property of approximate proportional equivalence.H l 0e ? 3380___PPT10.o30 p |,(  |d | c $P    | s *    In the coming slides, I will discuss this problem with you and present a new statistic that is appropriate to quantify model performance. The table shows three pairs of Mx and Px. The pairs have equal relative distance, while the absolute difference become progressively smaller. As you can see, the ALOR distance is approximately equal when reducing both probabilities using the same factor, and this equivalence increases as the probabilities become smaller. ,H | 0e ? 3380___PPT10.o40  ,(  d  c $P     s *    In the coming slides, I will discuss this problem with you and present a new statistic that is appropriate to quantify model performance. The table shows three pairs of Mx and Px. The pairs have equal relative distance, while the absolute difference become progressively smaller. As you can see, the ALOR distance is approximately equal when reducing both probabilities using the same factor, and this equivalence increases as the probabilities become smaller. ,H  0e ? 3380___PPT10.oxp^RЀ3ÿ lHb Xb? bW< ]P1`* 3cH!MT}`iʟ!!$;#"P+C!C)C"pWF0 =1 e328e(CC1Q~Pc= !"?$< ?g\h4s7# 0A_023!@#V ȱ`(+;@e7(ϳB2`Pܳ0 oЪ(,qI X\LR '=B&`TIId?v&Y} h7T obIFHe:Q.NHGs$g"$|3J\ KK2 b`p  3|THfC!v(d1lQ֒"uSz$&xY]lTE>wl-K%j۴hbJXM6b)vI1hKڤiMQ| 5!ф(iOJ1&PB9nif瞙3|o+&?f_"/ 3^Jj3XmfvvV.%ЉZ~KuR^B~eޒ1#x/n["gW)mkIyDb|r6,E )b)~(~Sze?p @T0aY,<%e5ǰ<;k-=v0cϹ|t^6,"Y>,QWcF`:./W%>m GG-tW,k1N{(=P`$ ~[Wૠ Ǡ_G-t큢ɬ}3iǟ/*%e>`2!_<(bHqR*+uV}6";Q&>sZirq͟N |,AWOnpt:hU=wqܳw^Tv]ЫEىRAjXrOC<3w7ٽI67M-QPʆ ؒ`Tg*"%I1nfvХATKmS-*AT|Ǥ"*}1hXsgw1t2{眹3sf|ߜM.}_9w!K/dKR d2,eSI/'ah(>@";gNgdLw?jxo앯=:aW4gU9S(q[""s?AAUopqЏ(CJx[X.:Íe /ikߨ kr컻:9226r0n;|?64 n%u_&A+g-A|dY:ZN/B⟗|G97AQ73LYV[N-Xi%hB g==``ڂ,Xzg y+`jaS:+LeA 4Z&sgO\{޵"SMB@ɔyR!h?4>7?N,K JiyG;>ZO< =^7Hٹ5Rcߨ"jL0Of"H3ѵR!z-uDw0+g4B?rƁ4{k͵t?ipGQ%ȥtJbZ|&._yݾXli&/ .'`{ &לJQ~!o43 l6<3 ۢVM&Y-V@aHFE|| Zj7*Π_X҇.ozgOjϦod K:02 t~SelunjȎ\(3{rqwI䈳\~SԸ켸,7BǜN7o]ҿ\PkY$_Musr[Y7wooʩ焿 ^5xMey9v D.U(qn oNMSe"ok^pn#5ǝ[pslJ,{Zoz_3o}*tg,{'Jo/E|pK3qv:"NT]ͯ Ua#- D9e8 tgoԷh}ydq$]v HD% OyOh+'0T hp   PowerPoint Presentationoweowe MVerduijn P36rMicrosoft PowerPointon@0&0L@@L9pGSg  )'    """)))UUUMMMBBB999|PP3f333f3333f3ffffff3f̙3ff333f333333333f33333333f33f3ff3f3f3f3333f33̙33333f333333f3333f3ffffff3f33ff3f3f3f3fff3ffffffffff3ffff̙fff3fffff3fff333f3f3ff3ff33f̙̙3̙ff̙̙̙3f̙3f333f3333f3ffffff3f̙3f3f3f333f3333f3ffffff3f̙3f3ffffffffff!___www4'A x(xKʦ """)))UUUMMMBBB999|PP3f3333f333ff3fffff3f3f̙f3333f3333333333f3333333f3f33ff3f3f3f3333f3333333f3̙33333f333ff3ffffff3f33f3ff3f3f3ffff3fffffffff3fffffff3f̙ffff3ff333f3ff33fff33f3ff̙3f3f3333f333ff3fffff̙̙3̙f̙̙̙3f̙3f3f3333f333ff3fffff3f3f̙3ffffffffff!___wwwpkeekekpeekkpkkeeeekkeeepekekeekkeeeeeekkpekkekeeepkkkeekkeeeeekkekkp޵ݵ``ffffffgmmm¼üݽݽݼݼfefffffmmϮݶݼݼݼ¼ݼ___ݼfffflgglmm޼ݼý޽…ݼݵffffffflgmmmϮϑݼݼݼ޼ݽ___ݼf`feffggϮϒݼ⼽⼽޽ݼ՜.+,08    On-screen Show n-s Times New RomanArial Wingdings Arial BlackVerdanaPixelMicrosoft Equation 3.0ADichotomization of ICU length of stay based on model calibrationICU length of stay (LOS)Main objectiveDataProblem of outcome definitionObjective of this study ApproachFirst results"Distances between probabilities I"Distances between probabilities I"Distances between probabilities IALOR distance#Distances between probabilities IIMALOR statistic!Procedure of threshold selectionAdditional results)Tree model for ICU LOS>5 days or deathDiscussion and conclusions Slide 19  Fonts UsedDesign TemplateEmbedded OLE Servers Slide Titles!_3 MVerduijnMVerduijn  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~      !"#$%&'()*+-./012356789:;@Root EntrydO)Pictures,#Current User4SummaryInformation(TPowerPoint Document(WDocumentSummaryInformation8,Root EntrydO),uD@Pictures,#Current User2SummaryInformation(T      !"#$%&'()*+-./0123@_3HunterHunter